Jump to content

I present a challenge.


Recommended Posts

@-- I don't think one would see any visual difference between the /(h)w/ and /w/ phonemes... like in the word pairs:

 

whine / wine

which / witch

etc.

 

They'll both use the same viseme pose. With my facial rig my goal is to make facial animations (lip-sync and expressions) easier and intuitive with the GUI I have planned.

 

With the approach @@eezstreet layed out in this challenge... the new lip_sync.gla will be appended on top of the _humanoid.gla using the head bone.

Link to comment

@-- I don't think one would see any visual difference between the /(h)w/ and /w/ phonemes... like in the word pairs:

 

whine / wine

which / witch

etc.

 

I agree. I think only when/win, whurry/worry, wheel/will are critical here.

 

They'll both use the same viseme pose. With my facial rig my goal is to make facial animations (lip-sync and expressions) easier and intuitive with the GUI I have planned.

 

With the approach @@eezstreet layed out in this challenge... the new lip_sync.gla will be appended on top of the _humanoid.gla using the head bone.

 

This sounds like like a fascinating project. I wish good luck to the both of you and I look forward to what you can create. If you need any help with testing, etc., count me in for additional support.

 

Will this project be freely available for others to integrate in their mods/projects once finished? I would be interested in seriously utilising it in an up-coming new game/mod I have planned, and in most of my cinematics work.

Link to comment
  • 2 weeks later...

Regarding the needed facial bones, I'd suggest taking a look at what Blizzard have done for their Orc:

 

O23QA9FZV1P91406741656210.jpg

 

As you can see, there's a decent amount of bones everywhere to give full control over the Orc's expressions. My point of showing this is to make sure that with your facial bones you have every area covered to give you 100% over expressions. :)

Link to comment

Yes... it will have all the necessary bones to drive fully-expressive facial expressions-- the 6 to 8 basic expressions... as well as be able to make custom facial animations for up-close cinematics. It will have a pose library where you can store custom facial poses.

 

As for the lip-synch'ing... the visemes will cover 44+ English phonemes... I also plan to build expandability for adding additional language viseme sets (others will have to make foreign language visemes that can be added to library of languages). I plan to use PRAAT for the sound analysis/phoneme identification-- already created the code to read in the phoneme list. PRAAT supports scores of languages.

Link to comment
  • 4 weeks later...

Yeah I saw that sometime back... when I loaded up the SoF2 GLA in ModView. It seems pretty crude... there are about 44 phonemes in the English language (varies by region and dialect) and they can be represented by about 20 visemes (...away from my PC right now so I can't recall the exact number) that I intend to create in the face rig. I think eye blinking should be treated separately from mouth visemes-- I think eye blinking should be in the facial expressions group (FACE_BLINK, FACE_WINK_R, FACE_WINK_L, etc.)-- or, create a separate "EYES_" group.

 

One thing I was thinking of was to be able to have the pupils dilate for close-up facial animations (they would be fixed/set to normal during gameplay-- but the feature would be there for close-up cinematics)... the eyes are what bring's a character to life in my opinion. :winkthumb:

Cerez likes this
Link to comment

That would work easily if carcass could process scale animation data. May have to code in an eye texture swap from normal to dilated.

Not sure if carcass would like scaling bones (even if linear)... guess we could do a test :winkthumb:

 

My idea was to make the Iris inner radius as 8 sided and use 8 helpers at the vertices to contract or expand the Iris revealing the curved pupil underneath.

 

I have this prototyped... and have made a test animation in 3ds Max.

Link to comment

That would work easily if carcass could process scale animation data. May have to code in an eye texture swap from normal to dilated.

Better idea - use an animmap and have the pupil dilation animate, but control the animation manually (JKG uses animmaps for the weapon indicators)

Cerez and DT85 like this
Link to comment

Better idea - use an animmap and have the pupil dilation animate, but control the animation manually (JKG uses animmaps for the weapon indicators)

Edit: I prefer that the animator have direct control of pupil dilation in the skeleton rig and simply have a default eye pupil dilation animation in the _humanoid.gla. Then in cutscene GLA's, they can quickly animate the eyes (not breaking their workflow) however they choose-- and not force them to learn Icarus scripting to deal with eye details externally in the game engine.

 

I'll post an animation of my prototype setup.

Link to comment

Animmaps, people. Animmaps. Animate the pupil using a shader and then manually control the pupil size using a SET_ value in ICARUS. ;)

Yes, that is certainly one way to do it... and I will create an example of that method in 3ds Max by animating a texture map.

 

And I also just thought of a way to do this by using only one (ah, ah, ah...) bone. (Did anyone catch my Sesame St. ref to "The Count"?)

 

I'll post example videos of each method.

Cerez likes this
Link to comment
  • 4 months later...

OK... I'm finally getting back to this while also trying to finish the 3ds Max dotXSI 1.x & 3.x file importer Maxscript (almost done-- struggling with skin weights!).  Anyways... for the facial rig you'll be able to control each area of the face to make unique expressions (could be used for custom facial.gla file for a cutscene cinematic); additionally to this overarching individual control, I plan to have six sliders that will allow you to pose the face quickly for the six universal facial expressions:  Joy, Sadness, Anger, Fear, Surprise, Disgust.  Within the 3ds Max Facial Rig GUI you'll also have the ability to create and store your own unique facial poses-- to be able to insert them anywhere along the animation timeline.  As for the six universal expression sliders, below is the progression of the emotional expression from a neutral face:

 

expressions_progression_zps3bb9af07.jpg

 

I'm still in the facial rig planning stages-- I intend to architect it so that the six expression sliders above can be combined/blended in an additive way-- e.g., you could have some amount of Disgust and Anger for a unique look... not every combination may result in believable expressions though.  I also hope to incorporate wrinkle-maps handled by shaders to go along with the facial expression (likely handled as animevents in the new facial.gla -- will have to work those details out with @@eezstreet, @@Xycaleth, and @@DT85).

 

...and I'll be working on the Jan Ors head mesh this weekend.  Any comments on the progression above?

Cerez and Rooxon like this
Link to comment
  • 1 month later...

It's been a while since I visited this thread. The last thing I remember was about the lips, and facial expressions as well. 

It is true, that both these and the body movement are in need of improvement.

 

In Deus Ex, which is far older than JA, the characters never looked static - one could notice them breathing for instance. Surely JA presented default animations, when an NPC would stand, doing nothing and look around occasionally. But it looks quite dull.

 

So what's the story now? Is there any chance of improving the animation, changing the skeleton and other stuff?

Cerez likes this
Link to comment

It's been a while since I visited this thread. The last thing I remember was about the lips, and facial expressions as well. 

It is true, that both these and the body movement are in need of improvement.

 

In Deus Ex, which is far older than JA, the characters never looked static - one could notice them breathing for instance. Surely JA presented default animations, when an NPC would stand, doing nothing and look around occasionally. But it looks quite dull.

 

So what's the story now? Is there any chance of improving the animation, changing the skeleton and other stuff?

 

Anything is possible now... I've gotten back to working on the Jan Ors head model and also laying out the concept for the Facial Animation/Lip-Sync GUI for 3ds Max.  When @@Xycaleth returns to working on the rend2 renderer, I've asked him to support wrinkle maps (normal maps that accompany certain facial expressions and blend with the base mesh normal map ). 

 

For example:  http://www.chrisoat.com/papers/Oat-Wrinkles%28Siggraph07%29.pdf

DT85 and Cerez like this
Link to comment

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
×
×
  • Create New...