Head/Face Animations

[Cerez]

Okay, I've started diving deeper into Icarus scripting, and I've come across something that left me a little baffled.

 

Using the "SET_ANIM_" set fields is clear enough when it comes to full featured animations (starting with "BOTH_"), but what about the smaller facial animations (the ones starting with "FACE_")?

 

If I try to apply any of these with the "SET_ANIM_UPPER" set field, I end up with the model stretching out into a factory default "T". Is there any way to preserve the existing body animation and overlay the facial animation on top of it?

 

Additionally, with an NPC_spawner entity the mouth of the model will not move when a "BOTH_TALK" is applied with "SET_ANIM_BOTH". What's up with that?

 

If anyone can shed light on either of these enigmas, I would really appreciate it.

[mrwonko]

You're presumably using some custom model with that NPC spawner? Then it's probably a shoddy one where the author just didn't bother to properly rig the mouth so it can't move.

[Cerez]

No, it's actually a good one, that has proper rigging. I've also tried to load the Jan model in its place into NPC_spawner and I got the same result with Jan. Could it be that these talk animations are expecting a sound file to work?

 

But my bigger worry is about the facial animations. I would like to get them working on the character's face without forcing the model to go into a "T" pose.

[Asgarath83]

Wait a moment, let me check my script of a mod level...

How i remember. you not need to use the Face animation

here there is a part of a cinematic script of a level of my mod. as you see, i use the SET_ANIM_BOTH for all animation of body and the set_face" parameters for the facial expressions.

 

set ( /*@SET_TYPES*/ "SET_FACEFROWN", -1.000 );
set ( /*@SET_TYPES*/ "SET_ANIM_BOTH", /*@ANIM_NAMES*/ "BOTH_STAND8" );
set ( /*@SET_TYPES*/ "SET_ANIM_HOLDTIME_BOTH", "-1" );
sound ( /*@CHANNELS*/ CHAN_VOICE, "sound/chars/termagent1/helena02.mp3" );
wait ( 7000.000 );
set ( /*@SET_TYPES*/ "SET_ANIM_BOTH", /*@ANIM_NAMES*/ "BOTH_STAND1" );
set ( /*@SET_TYPES*/ "SET_ANIM_HOLDTIME_BOTH", "-1" );
signal ( "XadoB" );
waitsignal ( "HelenaB" );
camera ( /*@CAMERA_COMMANDS*/ MOVE, $tag( "cam10b", ORIGIN)$, 0 );
camera ( /*@CAMERA_COMMANDS*/ PAN, $tag( "cam10b", ANGLES)$, < 0.000 0.000 0.000 >, 0 );
set ( /*@SET_TYPES*/ "SET_FACEFROWN", -1.000 );
set ( /*@SET_TYPES*/ "SET_ANIM_BOTH", /*@ANIM_NAMES*/ "BOTH_STAND5" );
set ( /*@SET_TYPES*/ "SET_ANIM_HOLDTIME_BOTH", "-1" );
sound ( /*@CHANNELS*/ CHAN_VOICE, "sound/chars/termagent1/Helena03.mp3" );
wait ( 17000.000 );
set ( /*@SET_TYPES*/ "SET_ANIM_BOTH", /*@ANIM_NAMES*/ "BOTH_STAND5" );
set ( /*@SET_TYPES*/ "SET_ANIM_HOLDTIME_BOTH", "-1" );
signal ( "XadoC" );
waitsignal ( "HelenaC" );
camera ( /*@CAMERA_COMMANDS*/ MOVE, $tag( "cam10", ORIGIN)$, 0 );
camera ( /*@CAMERA_COMMANDS*/ PAN, $tag( "cam10", ANGLES)$, < 0.000 0.000 0.000 >, 0 );
set ( /*@SET_TYPES*/ "SET_ANIM_BOTH", /*@ANIM_NAMES*/ "BOTH_STAND1_TALK3" );
set ( /*@SET_TYPES*/ "SET_ANIM_HOLDTIME_BOTH", "-1" );
sound ( /*@CHANNELS*/ CHAN_VOICE, "sound/chars/termagent1/Helena04.mp3" );
wait ( 8000.000 );
set ( /*@SET_TYPES*/ "SET_ANIM_BOTH", /*@ANIM_NAMES*/ "BOTH_STAND1_TALK2" );
set ( /*@SET_TYPES*/ "SET_ANIM_HOLDTIME_BOTH", "-1" );
sound ( /*@CHANNELS*/ CHAN_VOICE, "sound/chars/termagent1/Helena05.mp3" );
wait ( 12000.000 );
set ( /*@SET_TYPES*/ "SET_ANIM_BOTH", /*@ANIM_NAMES*/ "BOTH_STAND1" );
set ( /*@SET_TYPES*/ "SET_ANIM_HOLDTIME_BOTH", "-1" );
signal ( "XadoD" );
waitsignal ( "HelenaD" );
camera ( /*@CAMERA_COMMANDS*/ MOVE, $tag( "cam10b", ORIGIN)$, 0 );
camera ( /*@CAMERA_COMMANDS*/ PAN, $tag( "cam10b", ANGLES)$, < 0.000 0.000 0.000 >, 0 );
set ( /*@SET_TYPES*/ "SET_FACENORMAL", -1.000 );

 

[Cerez]

That's not a bad solution, but I can't see a SET_FACESMILE, for example, but there is a FACE_SMILE animation for it...

[Cerez]

Okay, I'm really stuck, and I need this awfully bad for my projects.

 

Could I ask a serious coder to help me out with this?

 

The animation I'm looking to apply are so called "head anims". They seem to be applied by the game in a random script -- would be good to know which -- by default on all cinematic stationary characters. The game does this somehow, it calls upon them, and I'd like to be able to do the same.

 

For cinematic projects it is crucial that the characters can express scripted emotions beyond the default set of frown and blink.

 

Their exact scripted names are the following:

 

FACE_ALERT

FACE_SMILE

FACE_FROWN

FACE_DEAD

FACE_TALK0

FACE_TALK1

FACE_TALK2

FACE_TALK3

FACE_TALK4

 

There is also a smirk animation somewhere used by the same default "stationary behaviour" character script. What I want to be able to do is to paste these head/face animations on top of an active body animation.

 

Could someone dig into the source and find out where this "stationary" character script is located, please? That would give us a clue how to apply the head animations.

[Cerez]

Here's a little something I've put together as an illustration of what I'm talking about:

 

 

The default JK animation scripts only allow a frown/angry and empty "blink" expression, but these other animations are there, just no-one has bothered to make them scriptable -- or so it seems.

 

I would really appreciate some help with getting these animations scriptable. It would add so much to the JK experience, and allow me to resume with my work.

[CaptainCrazy]

Hmmmm would it be possible to merely replace them ones, or maybe all of them, with custom facial animations instead?  I mean if you're making your own characters anyway then what harm could it do?

 

I just don't think that there's enough mesh there to work with.

[Cerez]

That's fine, but they still need to be scriptable. SET_FACEAUX, SET_FACEFROWN and SET_FACEBLINK are not enough! >.<'

[Asgarath83]

 @.

wait wait.

mmm

i have edited in my code the COLOR of FP_ABSORB, PROTECT by class of AI of character, the damage of weapons by class of NPC AI hitted,  and the efx of force power by class of shooter,. there are not perfect, and i need to debug again.

however, i never see the code about the facial animation, but for what i have understood, it's strongly very hard coded and difficult to edit.

I told you how make that into a scripts,. follow my example.

FACE_ALERT -> this animation see the face allarmed by a danger, i think is for the surprise, \ awake situation when the NPC see an enemy.

FACE_SMILE-> the NPC smile.

FACE_FROWN-> NPC is anger, worry, in combat FROWN it's the face of many combat animation.

FACE_DEAD -> Face for NPC death

 

If you set that's face by the BOTH_ANIM function with icarus you simpòly get the character in a "star pose" that make this expression, at the first of my scripting experience, i maked the same error.

there is another way: now i explain my scriopt step by step for making understandable to you of how it work.

 

set ( /*@SET_TYPES*/ "SET_FACEFROWN", -1.000 ); <- for infinite time the NPC get the face... frown. she is anger when tell this frace. this is the correct comand. if you use the animation command, you get the facial root pose,. there are 2 facial command inicarus, you need to use the specify facila command, not the animatioin commando, so the SET_FACEFROWN, SETFACE_AUX etc.

set ( /*@SET_TYPES*/ "SET_ANIM_BOTH", /*@ANIM_NAMES*/ "BOTH_STAND8" ); <- animation of character for the torso and legs.
set ( /*@SET_TYPES*/ "SET_ANIM_HOLDTIME_BOTH", "-1" ); <- during of animation in second. -1 durinbg the animation without ypu get a new ANIM_BOTH e ANIM_HOLDTIME command to the char or deactive BS_CINEMATIC.
sound ( /*@CHANNELS*/ CHAN_VOICE, "sound/chars/termagent1/helena02.mp3" );<- the voice. when a sound is played on the CHAN_VOICE or CHAN_VOICE_GLOBAL, the NPC that play the sound move his jaw bone on the bottom. more load it's the sound, more big is the movement, and the open of the mouth, the algorithm work in that mode, and cycle the other TALK pose. there are not random pose, they follow the Sound file pitch and waves. in tha'ts mode is realistic: if the sound is a screamn, the wave of soun file are load (you can see that in goldwave and audacity opening an mp3 or a wav sound with that programs) and the character use the FACE_TALK4 for the more strong part of the scream and FACE_TALK3 for the start and end of scream, so the character can open the mouth, "scream".  and close the mouth.

wait ( 7000.000 ); <- you need to set this paramter after played a sound by the CHAN_VOICE of the NPC because ... ehm, BEFORE you make the cinematics of a cutscene, you need to have the sound file with voice. you need to count how many second it's long.

for example,

"sound/chars/termagent1/helena02.mp3" <- my custom file is played in 7 second

so you need toi put in millisecond how many time the player execute the animation, the facial exprtession and the talking, the lenght

of that need to be the same of the audio file.

ANOTHER BLOCK: tha'ts is execture at the end of the voice: that set the character in the pose of listening the other character that answer to his words., that's is a dialogue script.

set ( /*@SET_TYPES*/ "SET_ANIM_BOTH", /*@ANIM_NAMES*/ "BOTH_STAND1" ); change stand animation
set ( /*@SET_TYPES*/ "SET_ANIM_HOLDTIME_BOTH", "-1" ); set duration of animation for undeterminate time
signal ( "XadoB" );<- tha'ts signal remind at another script where i set "waitsignal ("XadoB") as command, so that remind to the answer of "Xado" in the file of his taking dialogue, move and animation on this cutscene.
waitsignal ( "HelenaB" ); <- when xado stop to make the "xadoB"part of his dialogue, in the xado script there is the signal (helenaB)
camera ( /*@CAMERA_COMMANDS*/ MOVE, $tag( "cam10b", ORIGIN)$, 0 );
camera ( /*@CAMERA_COMMANDS*/ PAN, $tag( "cam10b", ANGLES)$, < 0.000 0.000 0.000 >, 0 ); <- the cam now change, and use another cam (you can make a cam in the map with a ref_tag named "cam1, cam2, cam3 etc, pointed an info_null direction with CTRL + K in radiant , that's is the direction where point the cam.
set ( /*@SET_TYPES*/ "SET_FACEFROWN", -1.000 ); <- whell, Helena is anger for the words of xado
set ( /*@SET_TYPES*/ "SET_ANIM_BOTH", /*@ANIM_NAMES*/ "BOTH_STAND5" ); helena change standing
set ( /*@SET_TYPES*/ "SET_ANIM_HOLDTIME_BOTH", "-1" );
sound ( /*@CHANNELS*/ CHAN_VOICE, "sound/chars/termagent1/Helena03.mp3" ); the answer of helena to xado.
wait ( 17000.000 ); the time of the anserer of Helena... very long XD i suppose that xado make she really angry XD

At this point, there another ANIMATION and ANIM_BOTH for helena pose and another signal \ waitsignal for the xado reaction at his voice.

If you want a NPC talk alone, you need to follow the sequence

Cam coordinates

facial exprtessiom,

NPC animation when talking

voice sound file played,

wait (number of second of the words. )

etc etc

 

if you got a dialogue with 2 NPC you need to make signal \ unsignal function you need that the 2 NPC run in the same time their spawnscript, (you need to make cinematic NPC spawned for the dialogue and removed and the end of the dialogue and the cutscene) and you need to set in "block"the dialog,.

so the dialog begin

XadoA->HelenaA->XadoB->HelenaB->XadoC->HelenaC etc,

On the spawnscript icarus file of xado you put thethe xadoa,b,c steps. on the helena files the helenabc, step.

i use this method because the method "affect xado blablabla" affect helena blablabla not ever work.

if the dialogue is with 3 or + characters... is MORE complicated too.

Scripting a dialogue and a cutscene is the mord hardest part of Icarus scripting.

:S

 

oi hope i explain you how function all that in understandable way. 

[Cerez]

Thanks for the break-down of your sample script, Asgarath. I appreciate the detailed explanation. Have you managed to make a character smile at another character's response yet? How about smirk? Or gape in surprise? That's what I mean about this serious limitation with the unscripted facial animations of the engine… It only really lets you make the character frown or stare, but it could do so much more! The necessary animations are there, they just needed to be scripted. Raven really messed up with this…

[CaptainCrazy]

I guess they just didn't need to use them facial expressions in their game(s).

[Asgarath83]

JKA have not much facial expression. :\ i am sorry but you can use only that. for surprise you can use an animation like BOTH_STAND9 (it's a pose of pre fight \ alerting) and the face_aux .

for the smirk... eh, unique is the face_smile. sorry

Otherwide you need to:

- build a new GLA file with new facial animation, i never make animation but it's not an easy work.

- change Icarus setting for include the new customanimation you builded.

- hope the engine recognize the customize facial animation scripted without a deep hacking of the code. in that i really cannot help ypu, sorry. the 2 character of example make a discusttion and the end of deal extract the sword and begin the fight, is a dialogue with a level boss. helena is a vampire hunter and xado... well,. xado is a raziel's son. lol.

[Cerez]

I guess they just didn't need to use them facial expressions in their game(s).

 

Which is one of the reasons the "acting" in the JK games is really awful. I think this should be a point of improvement for OpenJK for the sake of all JK SP mods and machinimas.

 

Otherwide you need to:

- build a new GLA file with new facial animation, i never make animation but it's not an easy work.

- change Icarus setting for include the new customanimation you builded.

- hope the engine recognize the customize facial animation scripted without a deep hacking of the code. in that i really cannot help ypu, sorry.

 

I had a look at the source code, and it would need to be changed as well in order for it to segment the new GLA correctly, or the new animations need to be matched exactly to the old GLA's timeline. About the your second point, how does one "change the Icarus setting" to include new animations made scriptable from the GLA file? Essentially this is the point I am failing to understand, and I would appreciate help from @@Serenity937, @@mrwonko, @@Xycaleth, @@redsaurus, @@ensiform, @@eezstreet, or someone from the OpenJK team who knows enough about this. Anyone?

 

Thanks.

[ensiform]

Which is one of the reasons the "acting" in the JK games is really awful. I think this should be a point of improvement for OpenJK for the sake of all JK SP mods and machinimas.

Sorry but not in the scope of OpenJK.

[Archangel35757]

Check out this thread:

 

http://jkhub.org/topic/4020-i-present-a-challenge/

 

...and this one:

 

http://jkhub.org/topic/4260-jan-ors-head-hi-def/

 

As soon as I finish the head, I will build a new facial rig that will support lip-synch''ing... having animated viseme poses that will represent the 44+ US-English phonemes (a single viseme may represent more than one phoneme).

[Cerez]

Sorry but not in the scope of OpenJK.

 

I thought one of the primary objectives of OpenJK was to fix bugs/mistakes made by the original Raven development team. I would consider these non-scriptable facial animations a big mistake that takes a lot away from the gameplay experience. I may be wrong, but I really don't see why this wouldn't be in the scope of OpenJK. The goal is to improve on the original product, is it not?

[Cerez]

Check out this thread:

 

http://jkhub.org/topic/4020-i-present-a-challenge/

 

...and this one:

 

http://jkhub.org/topic/4260-jan-ors-head-hi-def/

 

As soon as I finish the head, I will build a new facial rig that will support lip-synch''ing... having animated viseme poses that will represent the 44+ US-English phonemes (a single viseme may represent more than one phoneme).

 

Wow, this sounds impressive! This is way more than what I've had in mind. I've read up on it and I think this is a great plan and project. Maintaining backwards compatibility with the base game is crucial, though. We don't want to break the extensive library of existing community created mods.

 

Archangel/@@eezstreet, do you know how I would reference certain animations in the _humanoid GLA to make them scriptable in Icarus? Say I wanted to use the animation contained in FACE_SMILE and combine it with another existing body animation (e.g. BOTH_STAND1) through the script. Is this possible, and if not, why not?

 

The original game engine seems to do this at least internally with the idle behaviour animations on the models...

[Asgarath83]

I suppose you need to edit the Icarus files with the list of optionable parameters for the script (into the behaved folders) for add the new setting and condition to the list... and editing the Code part of ICARUS intercation for processing the new script command into the game.

I never make that, however, it's just a my theory. :\

[Cerez]

I suppose you need to edit the Icarus files with the list of optionable parameters for the script (into the behaved folders) for add the new setting and condition to the list... and editing the Code part of ICARUS intercation for processing the new script command into the game.

I never make that, however, it's just a my theory. :\

 

I've found where the Icarus interpretation code resides and editing the anims.h file is clear enough, although I'm not sure how I would reference the actual animation (animated frames) in the interpretation code.

 

The bigger problem is how to merge a FACE_ animation with a standard one (BOTH_). That's where the real trick lies...

[Asgarath83]

Mmm, try to see the code part of the FACE_AUX or FACE_FROWN icarus function, should be contain the merging by facial and torso animation or something like that.

[Cerez]

Mmm, try to see the code part of the FACE_AUX or FACE_FROWN icarus function, should be contain the merging by facial and torso animation or something like that.

 

You mean SET_FACEAUX and SET_FACEFROWN. Great suggestion! I failed to think of this! Thanks Asgarath, will do.

[Cerez]

Found the code responsible!

case SET_FACEFROWN:
		ent->client->facial_timer = -(level.time + holdtime);
		ent->client->facial_anim = FACE_FROWN;
		break;

Looks like the only way to get extra FACE animations in is to edit the engine code itself and re-compile it. That would explain why it hasn't really been attempted before.

 

But I'm tempted to do it...

 

Thanks so much, Asgarath! You really helped me here!

[Asgarath83]

My pleasure, @

i hope you have success with your modding projects.

[Cerez]

I'll be sure to let you in on them once I have something worthwhile.