Ragdoll and Model Transform Issues

[reeda1283]

Hi, so here is what I do to try and make sure my model matches the Ragdoll I have created (which I can see is being created and reacting to the world properly). I'll use the head as an example, as that is where I "begin" creating my ragdoll entities:

Code: Select all

Entity head = new Sphere(model.Bones["head"].Transform.Translation, .3f, 5); //Creates head entity at local position of model's head
Add(head, "head"); // Adds Entity to an array that will be called when player dies and ragdoll needs to be added to world
head.WorldTransform *= worldTransform; //adjust Entity to proper world transform (this is the world transform of a charactercontroller that is already in space)

Later on (still in initialization), I bind the offsets of the model bones to an entity:

Code: Select all

BindTransforms = new Matrix[model.Bones.Count];
                
                for (int i = 0; i < model.Bones.Count; i++)
                {
                    string name = model.Bones[i].Name;
                    switch (name)
                    {
                        case "head":
                            RagdollBones.Add(new RagdollBone(name, GetOffset(model.Bones[i].Transform,  //the offset function returns Matrix.Invert(model.bones[i].Transform * head.WorldTransform
                                head.WorldTransform), head));
                            break;
               ............

Finally, I populate my BindTransforms matrix during draw calls (which is provided to my skeletal animation shader)

Code: Select all

for (int i = 0; i < model.Bones.Count; i++)
            {
                string name = model.Bones[i].Name;
                foreach (RagdollBone bone in RagdollBones)
                {
                    if (name == bone.Name)
                    {
                        BindTransforms[i] = bone.Offset * bone.RagBone.WorldTransform;
                    }

I invariably get bodyparts stretched through the world

ps I know the code is ugly and inefficient, but Im just trying to get a working case down first;

[Norbo]

This is more of a graphical issue than a physics issue, but my general recommendation for this kind of thing is to simplify it into individual pieces and then do an isolated test on those little pieces. Run a debugger through the little tests, checking the values against your expectations. When the values you see stop making sense, you've probably found the problem or at least a hint about the problem

[reeda1283]

Yea, I typically don't post unless I'm getting to my wits end I guess a simpler question would be, does the below approach make sense?

In ragdoll setup:
1. Create Head entity at Local Transform of the model's head bone.
2. Multiply new entity's transform by the world transform of the actor to get actual world position.
3. Grab "offset" of bone to entity by taking inverse of bone's transform and multiplying it by the world transform (I will eventually do this in step 1)

During Draw
4. Return each bones local transform by taking multiplying the above mentioned offset by the world transform (my shader, like most, takes a single world matrix as a separate input so this is necessary).

[Norbo]

I'd do something like this:

-For updating ragdoll entities from graphical bones:
1) Compute the current world transform of the graphical bone in actor space.
2) Multiply the actor space transform by the actor's world transform, store it in a temporary instead of transforming the bone transform itself because the shader presumably takes the actor transform and applies it to the graphic.
3) Multiply the local offset from bone to entity (defined at content creation time) by the computed world transform. That's the entity's world transform.

-For updating graphical bones from ragdoll entities:
1) Multiply the inverse of the bone-to-entity relative transform (the one defined at content creation time) by the entity world transform to get the world space bone position.
2) Multiply your graphical bone world matrix from #1 by the inverse actor world transform and pass in the actor world transform to the shader.

Watch out for multiplication order, and note that passing in Matrix.Identity for the shader actor transform and just putting all the bones into world space is an option.

[reeda1283]

OK, I think I see where I may be missing a step:

1) Compute the current world transform of the graphical bone in actor space.

So, I guess this means if I just take the model's Bones["head"].Transform, that is NOT actually in actor space (sheesh I should know this for how much work I've done), and that I need to figure out what it IS in actorspace?

So I guess I would need to do something like the below?

foreach (ModelBone bone in model.Bones)
{
Matrix BoneActorTransform = bone.Transform;
ModelBone parent = bone.Parent;
while (parent != null)
{
BoneActorTransform *= bone.Transform
parent = parent.Parent;
}
}

[Norbo]

If you don't already have it computed, you will need to take into account the chain of relative transforms to get the actor space bone transform, that's correct. That particular chunk of code has a few issues though

[reeda1283]

I feel I might be a victim (or benefactor) of the old adage:

"Give a man a fish and you feed him for a day. Give a man a fish and you feed him for a lifetime."

It would be awesome if you could point out my failings though.

[Norbo]

Alright, alright- I'll give you one, but I'll leave whether or not there are other issues vague to keep you on your toes

As is, it is computing bone.Transform^n, where n is the number of bones traversed from child to the root.

[reeda1283]

Ok, so as far as getting bones to map to entities, I have actually figured this out, and was able to use the technique employed by the XNA to sorta get things working. However, I still have a slew of issues. I can take these over to the GameDev forums if you feel they are too non-BEPU related.

1. Like the XNA example, I am using a root transform as well as an Matrix array for skin transforms aka when I set the models world, I use: actualOffset * rootTransform * worldMatrix. When ragdoll is happening, I am currently setting rootTransform to Identity, since the bones should be driving all animations. I am not using any offset when ragdolls are controlling the model. Are these assumption correct?
2. What World matrix should I pass into the shader? I am concerned that my "ActorWorld", which is from the character controller that controls the model when it isn't Ragdoll, is dependent on an offset, which is causing separation of model and entity to occur when rotation happens. See
3. I am using your method to apply the offset when bone movement is being calculated, but can you confirm this is how I should be generating those offsets at compile time?
Matrix offset = Matrix.Invert(bindPose[bindPoseIndex]) * Entities["body"].WorldTransform

I am amazed at how much trouble this is giving me :/ Thanks for any help you can provide.

[Norbo]

I would recommend putting it over on a more general forum. At least on gamedev, you aren't just waiting on one guy to give an answer- and who knows, if I get some time, I might answer over there too

[reeda1283]

Cool, I just did that. This whole situation is hurting my brain!

[reeda1283]

Soooo, I made some progress, but am struggling with the last (and possibly most important) item of getting the bones to properly match the entities.

Since I have kind of gotten crickets over on GameDev, maybe you could have a look?

http://www.gamedev.net/topic/633962-rag ... ng-issues/

[Norbo]

Sorry, I don't have enough time for a full thorough analysis, but it sounds like system complexity is getting in the way of solving any core math issues present. I would recommend isolating things down to just single transform at a time. Sometimes (frequently (nearly always)), doing a bit of extra work to isolate issues into tiny little toy test cases with limited scope is worth it.

If you can incrementally build the system from the ground up through individually tested and individually understood pieces (conceptually or with actual separate implementation modules), the resulting system is typically much, much easier to understand and much less prone to bugs. Any bugs remaining are typically easier to chase down and arise from the interaction from the pre-verified subcomponents.

[reeda1283]

Since I STILL can't seem to figure this out on my own, below is a simpler recap of what I'm doing. Hopefully someone will have an idea where I may be misstepping from this more boiled down process.

First, I grab the offsets of key bones of my model to their associated physics entities at initialization time. For the ragdoll structure, I'm using the setup from the ActionFigure example. Example:

Code: Select all

 bodyOffset = Matrix.Invert(RelativeBindPose["hips"]) * body.WorldTransform

Fast-forward to the update function for the ragdoll, which just generates the final skinning matrices for the shader. First I get all transforms for individual bones, like so:

Code: Select all

 if (boneIsAttachedtoPhysicsEntity)
        boneTransform = bodyOffset * associatedEntity.WorldTransform
    else
        boneTransform = relativebindPose[bone]

Finally, calculate the skinning matrices in local space, which are then passed to the shader. The root of course has no parent, so its worldTransform ends up being the same as the boneTransform

Code: Select all

worldTransform = boneTransform * parentWorldTransform
    skinTransform = Matrix.Invert(relativeBindPose) * worldTransform

This works fine for the first joint (pelvis), but the bodyparts get warped as I move down the heirarchy. I think my order of operations in calculating the offset (and/or applying it at the end) could be suspect, but after trying other options, I think there is something fundamental I am missing.

[Norbo]

bodyOffset = Matrix.Invert(RelativeBindPose["hips"]) * body.WorldTransform

Let's derive the source of this expression. ActorSpaceBoneToEntity below is the transform from the bone transform to the entity transform in actor space at initialization.

InitialBoneTransform * ActorSpaceBoneToEntity = InitialEntityTransform
(InitialBoneTransform^-1 * InitialBoneTransform) * ActorSpaceBoneToEntity = InitialBoneTransform^-1 * InitialEntityTransform
ActorSpaceBoneToEntity = InitialBoneTransform^-1 * InitialEntityTransform

So, assuming that RelativeBindPose["hips"] returns the initial actor space absolute transform of the hips bone (this is a critically important assumption! if it's a relative transform from parent to child, none of this or the original will produce a particularly meaningful result) and the InitialEntityTransform is in actor space (the spaces must be consistent for this process to make any sense), the given code produces the ActorSpaceBoneToEntity. However, the ActorSpaceBoneToEntity isn't necessarily useful by itself.

To demonstrate this, note that the ActorSpaceBoneToEntity was 'built' to be used like entityTransform = boneTransform * ActorSpaceBoneToEntity. This works when given the initial transforms, but watch what happens if you try to use a different bone transform. ActorSpaceBoneToEntity is in actor space, not the local space of the bone. Instead of getting the entity position, it just applies that same relative transform unhelpfully.

If we look at the usage of the ActorSpaceBoneToEntity (bodyOffset):

Code: Select all

boneTransform = bodyOffset * associatedEntity.WorldTransform

we can see that this produces an even less meaningful result. Using the naming convention adopted earlier exposes the issue:

BoneTransform = ActorSpaceBoneToEntity * EntityTransform

Just reading it off in English doesn't sound right. But, this usage does hint at the true desired value of bodyOffset. For that expression to make sense, it should read:

BoneTransform = EntityToBoneInEntityLocalSpace * EntityTransform

By making bodyOffset the offset from the bone to the entity in the entity's local space, you can reconstruct the world transform of the bone if you are given the entity's world transform. Think of it as attaching the offset to the entity; the offset 'stick' is attached to the entity spins and moves with the entity. The end of the stick is where the bone is.

We want to compute that stick. The above can be adapted quite directly and then solved for an expression for EntityToBoneInEntityLocalSpace:
InitialBoneTransform = EntityToBoneInEntityLocalSpace * InitialEntityTransform
InitialBoneTransform * InitialEntityTransform^-1 = EntityToBoneInEntityLocalSpace * (InitialEntityTransform * InitialEntityTransform^-1)
EntityToBoneInEntityLocalSpace = InitialBoneTransform * InitialEntityTransform^-1

Another way to think of this is that the InitialBoneTransform is being 'pulled' into the local space of the entity by multiplying by the initial entity transform inverse. This is the same way cameras work: by multiplying by the camera's view matrix, you are actually multiplying by the inverse of the camera's world matrix. That 'pulls' objects from world space into camera space (also known as view space).

So, using BoneTransform = EntityToBoneInEntityLocalSpace * EntityTransform with the above construction of an EntityToBoneInEntityLocalSpace transform gets you the BoneTransform in world space (assuming the EntityTransform is in world space).

Be sure to keep track of what space things are in; passing something from world space into a function expecting actor space would be nasty. The below snippet is suspect:

Code: Select all

if (boneIsAttachedtoPhysicsEntity)
boneTransform = bodyOffset * associatedEntity.WorldTransform
else
boneTransform = relativebindPose[bone]

As just described, boneTransform = bodyOffset * associatedEntity.WorldTransform will produce a world space bone transform (assuming associatedEntity is being simulated in world space, and not actor space somehow). However, relativebindPose seems like it will almost certainly not contain a world space bone. Instead, (consistent with the earlier previous assumption) I'd assume it contained an actor space bone transform- or, based on the name, a relative transform from a parent bone to a child bone (which is even less appropriate when compared against the boneTransform = bodyOffset * associatedEntity.WorldTransform computed above). That inconsistency will cause problems.

If we look ahead:

Finally, calculate the skinning matrices in local space, which are then passed to the shader. The root of course has no parent, so its worldTransform ends up being the same as the boneTransform
worldTransform = boneTransform * parentWorldTransform
skinTransform = Matrix.Invert(relativeBindPose) * worldTransform

it appears the previous assumptions are violated. 'boneTransform' is here is used as if it is a local transform from the parent to the child. That is not what was computed by the assumptions used above. Using the stated assumptions, you do not need to multiply by the parentWorldTransform at all. The boneTransform is the world transform (when computed as BoneTransform = EntityToBoneInEntityLocalSpace * EntityTransform).

There are a variety of ways to make this consistent again. For example:
-You could change up the assumptions so that it tries to work with relative transforms earlier on.
-You could compute BoneTransform = EntityToBoneInEntityLocalSpace * EntityTransform and then pull BoneTransform into the local space of its parent transform so that BoneTransform would then be a relative transform (this is somewhat circular). The boneTransform grabbed directly from relativebindPose when the bone isn't following an entity could be left alone then.
-You could change up the final step to expect world space data, or at least actor space data. Whichever way, make sure the spaces are consistent.

So, there's no one spot where the problem is. There just appear to be various inconsistencies depending on how the system is looked at.

Those sorts of inconsistencies cannot really be debugged by looking only at the final output of a long sequence of transforms- this is where the value of breaking down the problem comes in. Instead of fiddling with different parts of the long pipeline, work with only a single part at a time- like the construction of the initial 'stick' transform to ensure that it is producing exactly what you want. Doing so is hugely easier. Coding additional debug visualization to help view these intermediate stages is almost always worth it if problems are encountered.

In fact, this is so important, it's worth a completely redundant paragraph: in the presence of nontrivial failures, do not attempt to debug the final result of a complex system all at once. It will only cause suffering. Too many things can fail in too many ways. Don't be afraid to make a completely separate program to isolate a single tiny little piece, or to test your understanding of some of the individual fundamental processes without any interference from other parts of the system. It is one of the best possible uses of time in debugging (hence 'unit testing,' 'test driven development,' and all the related concepts).

(Disclaimer: This was written immediately before snooze time, and, as it is now snooze time, proofreading will be nonexistent. There may be catastrophic errors in the above.)