Bug: centerOfMassOffset translates object's centerPosition

Discuss any questions about BEPUphysics or problems encountered.
bryanedds
Posts: 14
Joined: Tue Jul 13, 2010 6:17 pm

Bug: centerOfMassOffset translates object's centerPosition

Post by bryanedds »

I'm having a strange bug. For some reason, setting an entity's centerOfMassOffset translates it's centerPosition. This is a problem because I serialize my object's centerOfMassOffset and centerPosition. Because I set and save the centerOfMassOffset to (0,-5,0), the object is translated upward (0,5,0) on the first frame after being deserialized. I tried hacking in a fix like this -

Code: Select all

        public Vector3 CenterOfMassOffset
        {
            get { return entity.centerOfMassOffset; }
            set
            {
                Vector3 comoDelta = entity.centerOfMassOffset - value;
                entity.centerPosition += comoDelta;
                entity.centerOfMassOffset = value;
            }
        }
- but since the entity.centerPosition is a zero vector no matter what is written to it before the first physics frame, it doesn't work. I cannot figure out any reasonable workaround.

It really seems like centerOfMassOffset affecting the object's centerPosition is a bug. As a small note, all deserialization happens before the first physics frame is processed (and must for obvious reasons).

Any ideas?
Norbo
Site Admin
Posts: 4929
Joined: Tue Jul 04, 2006 4:45 am

Re: Bug: centerOfMassOffset translates object's centerPosition

Post by Norbo »

The centerOfMassOffset should not affect the centerPosition, as you correctly assume. It does not directly touch the centerPosition when set; I think what might be happening is an awkward order of writes.

The centerPosition, orientationMatrix, orientationQuaternion, centerOfMass, etc. are buffered properties (hence the unchanging value); the centerOfMassOffset is not buffered. It's very easy to create a situation where the sequence of reads and writes produce an undesired result (like in that example where the read value of centerPosition is zero, resulting in the entity being positioned at the delta after the first frame regardless of where it was loaded initially).

Unless you need the threading safety, I'd recommend switching the trouble spots to use the internal-prefixed properties like internalCenterPosition. These directly access internal values as opposed to working through buffers.

I've come to heavily dislike the entity interface. Almost everyone stumbles over this buffering issue at some point (I know I have!). The buffered properties are implied by the design to be the default, suggested approach, when in reality, the 'internal' properties are far more likely to fit better in most situations. I'm going to have to make a nasty breaking change in some upcoming major version where the centerPosition adopts the meaning of the internalCenterPosition and loses its buffered nature. Buffered properties will be pushed into a suitably marked container, so that people that need thread safety will know to use it, but others will stay away.
bryanedds
Posts: 14
Joined: Tue Jul 13, 2010 6:17 pm

Re: Bug: centerOfMassOffset translates object's centerPosition

Post by bryanedds »

Being the author of Ox, I know how pernicious one's own API flaws are. My sincere sympathies :)

I did try transposing the order in which the centerPosition and centerOfMassOffset are serialized, but the result is the same. So it seems the write order does not appease.

Then I tried using the 'internal' variant of 'centerPosition' (and all the other properties that provide them), but this does not rectify the behavior. I looked for 'internalCenterOfMassOffset', but there's isn't one. Did you forget to add the 'internalCenterOfMassOffset' property?

What happens if I need to use the threading features of BEPU? Since transposing the serialization of centerPosition and centerOfMassOffset does not fix the issue, am I unable to use the threading features at all?

Finally, I use the Vehicle and Wheel classes to do a lot of things. It does not expose any 'internal' properties. So I'm concerned that it forwards its property values to buffered properties. This seems to yield inconsistent property semantics. For example, setting a vehicle's position and orientation using the exposed 'internal' properties does move its body to the right place before the first physics frame. However, it's wheels remain at the world origin until after the first frame. This is an issue since the editor has a common usage mode where physics are not processed (Pause Physics Mode). In this mode, I can only drag the shell of the car as neither the wheels nor the (most importantly) bounding box will update. For now, I just have to disable all dragging in the view and disable all transformation properties in the property grid while in Pause Physics mode. A real shame.

I know I brought up a lot of issues, but please try to address each one as best as you can. They are all quite critical :)

Thank you very, very much for your time! :D
Norbo
Site Admin
Posts: 4929
Joined: Tue Jul 04, 2006 4:45 am

Re: Bug: centerOfMassOffset translates object's centerPosition

Post by Norbo »

I did try transposing the order in which the centerPosition and centerOfMassOffset are serialized, but the result is the same. So it seems the write order does not appease.
I believe there is something hiding somewhere, obfuscated by the unintuitive properties. For example, this tiny piece of code works as expected:

Code: Select all

            Box box = new Box(Vector3.Zero, 1, 1, 1, 1);
            //Technically, the internalOrientationMatrix is equivalent to the orientationMatrix property here, since 
            //box does not yet belong to a space.  When an entity does not belong to a space, buffered properties
            //act like unbuffered properties.  I don't like this inconsistent behavior either.
            box.orientationMatrix = Matrix.CreateFromAxisAngle(Vector3.Normalize(new Vector3(1, 1, 1)), 46.1245267f);
            box.centerOfMassOffset = new Vector3(1, 1, 1);
            space.add(box);
            //Now that the box belongs to a space, this value will be put into the write buffer to execute at the first frame.
            box.centerPosition = new Vector3(0, 1, 0);
            //If I check the centerPosition value here, it is (0,0,0).  So is internalCenterPosition.

            ...

            //In the update method:
            //Still zero values before update.
            space.update(dt);
            //Both values now (0,.999..., 0).
            //(Slightly less than (0,1,0) since it fell a bit in one frame).
There may be some legitimate write/read order that produces incorrect results, but I haven't encountered it yet. A reproduction case would help squish the bug if it exists.
I looked for 'internalCenterOfMassOffset', but there's isn't one. Did you forget to add the 'internalCenterOfMassOffset' property?
An internal version of that particular property would be redundant, because it is unbuffered to begin with. The confusion is understandable, given the names ;) Each property's associated XML documentation lists whether or not it is buffered/interpolated.
What happens if I need to use the threading features of BEPU? Since transposing the serialization of centerPosition and centerOfMassOffset does not fix the issue, am I unable to use the threading features at all?
All of the BEPUphysics internal threading can be used without worrying about using thread-safe properties (for the most part), since the engine follows a fork-join model where all threads are idle by the time the main thread exits the Space update. The only time thread-safe properties become truly useful is when the physics update loop is set up on a different thread and individual values need to be accessed asynchronously, which is quite rare.

To safely read more than a single entity's buffered property while the engine is running asynchronously (in rendering, for example), use the space's motionStateManager's getMotionStates method. This method handles buffer locking and atomically reads from a single frame, avoiding reading one entity's state from frame X and the next entity's state from frame X + 1.
Finally, I use the Vehicle and Wheel classes to do a lot of things. It does not expose any 'internal' properties. So I'm concerned that it forwards its property values to buffered properties.
None of the properties on the Vehicle are buffered in the same way entity properties are (i.e., they are not thread safe). However, various world space properties are updated as needed during a space update. The most important one for rendering is the wheel shape's worldTransform property. This is called at the end of each frame. Fortunately, the Vehicle class exposes the updateAtEndOfFrame method publicly, so you can call it whenever you want to force an update of the vehicle's wheel transformations.

The fact that nothing is buffered on the latest version of the vehicle is actually an oversight. The world transform should be protected in some way to prevent ugliness when running asynchronously. I'll take a look at this when I can.
In this mode, I can only drag the shell of the car as neither the wheels nor the (most importantly) bounding box will update.
Entities have a forceBoundingBoxRefit method that can be called. It will not automatically update collision pairs, though. The space's broadphase exposes an updateCollisionPairs method if needed. The upcoming rewrite will also make it easier to use the other stages, like narrow phase collision detection, externally.
This is an issue since the editor has a common usage mode where physics are not processed (Pause Physics Mode).
I would like to make this kind of usage pattern easier to deal with. The special cases involved with buffered properties, vehicle wheels, bounding boxes and similar things make it a bit unwieldy. One option may be to allow timeScale to smoothly slide all the way down to 0, while still updating the necessary parts of the engine. Right now, setting timeScale to 0 results in a lot of nasty divide-by-zero issues, but there may be a way to work around all of them.

By the way, others making editors found use for the space's forceBufferedStateUpdate and forceWriteBufferFlush methods. These allow buffered entity properties to be used without needing a whole space update.
Norbo
Site Admin
Posts: 4929
Joined: Tue Jul 04, 2006 4:45 am

Re: Bug: centerOfMassOffset translates object's centerPosition

Post by Norbo »

One important fact I forgot to mention:
Buffered properties' read values also include an interpolation component if internal time stepping is enabled. The interpolation allows smooth movement when frame updates are occurring at unpredictable intervals (when internal time stepping is enabled). That's their main use outside of thread safety.

The way all of those features are bundled together bugs me to no end.
bryanedds
Posts: 14
Joined: Tue Jul 13, 2010 6:17 pm

Re: Bug: centerOfMassOffset translates object's centerPosition

Post by bryanedds »

I'm still working on using your suggestions. Gonna be awhile before I try all the different transmutations.

However, I immediately noticed that mutating the vehicle body's 'internal' properties leads the wheels to drift off of the car, especially when the car is flying off a cliff. There just seem to be too many anomalies when using the internal properties. I'm a little afraid to use them now. Perhaps it is best that they're not the default after all...

I have gotten dragging the car with physics disabled to work a little better with calls to space.forceBufferedStateUpdate(). I presume I'm supposed to call space.forceWriteBufferFlush() near it, but I can't figure out whether, why, or what order I should be calling it. What is the semantic difference between "forcing all entity buffered states to be immediately updated to reflect internal values" and "forcing all buffered entity state writes to be immediately applied"?
Last edited by bryanedds on Thu Jul 29, 2010 11:24 pm, edited 1 time in total.
Norbo
Site Admin
Posts: 4929
Joined: Tue Jul 04, 2006 4:45 am

Re: Bug: centerOfMassOffset translates object's centerPosition

Post by Norbo »

However, I immediately noticed that mutating the vehicle body's 'internal' properties leads the wheels to drift off of the car, especially when the car is flying off a cliff.
If the engine isn't being updated asynchronously, setting internal properties before an update should yield the same results as setting buffered properties (provided that the end result in the write buffer is the same as what was written to the internal property).

If the internal properties are written after the space updates but before the draw, and the draw bases itself on the internal properties, then the wheels will indeed appear to drift. This is because the wheels' transforms were updated at the end of the frame, before the writes. Writing to buffered properties after a space update won't affect the rendering since the writes won't be applied until the beginning of the next frame (unless the buffer is manually flushed).

Wheel drift could also occur when updating asynchronously due to the wheels possibly being desynchronized from the body. It would probably look very jittery.
There just seem to be too many anomalies when using the internal properties. I'm afraid to even use them now.
Don't get scared, they really are the simplest form of properties despite their ominous name! :D As long as the engine isn't running asynchronously, they are the most direct and effective way to manage your entity's state. When I create demos or tests, I generally use internal versions simply to avoid the lack of buffered value updates which wreak havoc when you're doing calculations that are dependent on buffered properties.

If internal timestepping is off and the engine isn't updating asynchronously, you can make an entire game without ever referencing a buffered property once. In fact, using buffered properties in such a game would most likely be a mistake and cause bugs. If the game uses internal timestepping to achieve framerate independence, then the interpolation component of the buffered value becomes useful for rendering. I'd still use internal properties for all the rest of the logic, however.

Asynchronous updating makes everything an order of magnitude more difficult (for more reasons than just buffered properties), so I generally advise against it. But, in some cases, it has a use, and in those cases, the buffered properties' thread safety makes sense. It still requires a lot of care to avoid corrupting everything with weird read/write orders, though.
Perhaps it is best that they're not the default after all...
It might not seem like it, but I'm still fairly sure the buffered properties are the evil beasts here, framing the poor innocent internal properties :)
What is the semantic difference between "forcing all entity buffered states to be immediately updated to reflect internal values" and "forcing all buffered entity state writes to be immediately applied"?
The asynchronous updating documentation has some pictures that might help:
http://www.bepu-games.com/BEPUphysics/d ... tation.pdf

Basically, when you update buffered states (at the end of a frame or when you force it manually), it locks the engine to stop any mid-snapshot space updates, takes a snapshot of the entities' values, stuffs them into a 'read' backbuffer, and then flips that buffer to the front 'read' buffer. Buffered reads always see the values in that front read buffer.

Writing to buffered properties enqueues entries into a write buffer. When the write buffer is flushed (at the beginning of a frame, or when you force it manually), those write buffer entries get applied and become the new 'internal' state of the engine.

As a thought experiment to apply these concepts, imagine you wrote a bunch of stuff to the buffered properties. It gets enqueued into a write buffer, but not applied until the write buffer is flushed. Imagine you forced a buffered states update before the write buffer flushed; now, the read buffer contains the most recent internal values, but not any of the values that were in the write buffer. If you then flushed the write buffer, the read buffers would be out of date compared to the new internal values.
bryanedds
Posts: 14
Joined: Tue Jul 13, 2010 6:17 pm

Re: Bug: centerOfMassOffset translates object's centerPosition

Post by bryanedds »

ASIDE: One thing I'll not do with BEPU (or any physics engine) is process physics while rendering (or whatever). I'm happy as long as BEPU is using multiple cores during its own isolated frame update. Additionally, I like the additional cores being free for my own task partitioning (who's to say some rendering (or other engine) code won't be able to make use of multiple cores?).
Norbo
Site Admin
Posts: 4929
Joined: Tue Jul 04, 2006 4:45 am

Re: Bug: centerOfMassOffset translates object's centerPosition

Post by Norbo »

ASIDE: One thing I'll not do with BEPU (or any physics engine) is process physics while rendering (or whatever). I'm happy as long as BEPU is using multiple cores during its own isolated frame update. Additionally, I like the additional cores being free for my own task partitioning (who's to say some rendering (or other engine) code won't be able to make use of multiple cores?).
That's good and makes things a lot easier for both of us :)

I tend to agree and like designs where each module has full control over the resources available. It simplifies things dramatically and encourages finding ways to scale individual components to match multicore advancements without relying on creating more asynchronous tasks (AI, physics, etc.).
bryanedds
Posts: 14
Joined: Tue Jul 13, 2010 6:17 pm

Re: Bug: centerOfMassOffset translates object's centerPosition

Post by bryanedds »

Norbo wrote:
However, I immediately noticed that mutating the vehicle body's 'internal' properties leads the wheels to drift off of the car, especially when the car is flying off a cliff.
If the engine isn't being updated asynchronously, setting internal properties before an update should yield the same results as setting buffered properties (provided that the end result in the write buffer is the same as what was written to the internal property).

If the internal properties are written after the space updates but before the draw, and the draw bases itself on the internal properties, then the wheels will indeed appear to drift. This is because the wheels' transforms were updated at the end of the frame, before the writes. Writing to buffered properties after a space update won't affect the rendering since the writes won't be applied until the beginning of the next frame (unless the buffer is manually flushed).
Given that I'm eschewing asynchronous updates and using the 'internal' properties where exposed, am I forced to have three different updates per frame - logic update, bepu physics update, then sync visuals update? Creating the new seam through the existing logic update is possible, but it does double the amount of time spent traversing the scene with updates calls. Not a big deal, just seems a little weird to me right now.
Norbo
Site Admin
Posts: 4929
Joined: Tue Jul 04, 2006 4:45 am

Re: Bug: centerOfMassOffset translates object's centerPosition

Post by Norbo »

I generally prefer designing it so it does indeed follow the "physics affecting logic->space update->graphics update" outline. The physics affecting logic is usually things like updating the steering angle of vehicles from user input.

It's perfectly possible to use a different pipeline, like "space update->logic and graphics update," or even doing some logic on each side of a space update. Care has to be taken to ensure that things stay consistent regardless- doing logic on either side might make it a little harder to keep track of, though.

Using the BEPUphysicsDemos as an example, it makes sense to do logic like setting up a ball launch before a space update so that the current frame's space update will send the ball on its way. Yet in the demos, the grabber raycast is also performed before the space update. Technically, this is using the last frame's positions snapshot since the space hasn't updated during this frame. I could push the grab so that it is tested after the space update, but then the current frame wouldn't apply any force. It turns out that it doesn't matter from the user's perspective in this case. Either way would be acceptable.

As long as things are handled consistently, the worst case result is that some things take effect one frame behind or use data that's one frame out of date. The problems come when something is interpreted inconsistently.
bryanedds
Posts: 14
Joined: Tue Jul 13, 2010 6:17 pm

Re: Bug: centerOfMassOffset translates object's centerPosition

Post by bryanedds »

Okay, I'll put in the seam and see how it goes :)

I will try to find a way to reproduce the center of mass offset bug for you as soon as I can!

Many thankings upon you my Norbo friend for all your help ;)

cheers~!
Norbo
Site Admin
Posts: 4929
Joined: Tue Jul 04, 2006 4:45 am

Re: Bug: centerOfMassOffset translates object's centerPosition

Post by Norbo »

I should probably also describe in more detail what might be encountered in a "space update->logic and graphics update" pipeline, since it is a valid option.

So if the idea is:
-The space updates, computing the new internal values and updating buffered states to match.
-Physics-affecting logic runs, presumably changing position or orientation.
-Graphics update bases itself on the latest physics data.

One issue can be:
-The space updates, computing the new internal values and updating buffered states to match.
-Physics-affecting logic runs, presumably changing position or orientation.
*Writes to INTERNAL values.
-Graphics update bases itself on the latest physics data.
*Reads from BUFFERED/INTERPOLATED values.

In the above, the physics-affecting logic is ignored by the graphics update since the buffered values haven't been changed by the writing to the internal values. You could update the buffered states so that the graphics sees the latest version, or just ignore it and let the engine deal with it next frame. If internal time stepping is disabled, you can also just read internal values for your graphics.

Another issue can be:
-The space updates, computing the new internal values and updating buffered states to match.
-Physics-affecting logic runs, presumably changing position or orientation.
*Writes to BUFFERED values.
-Graphics update bases itself on the latest physics data.
*Reads from any values.

In this version, the physics-affecting logic is ignored by the graphics update since the buffered values haven't been written yet. You can either flush the write buffer and read internal states for graphics, or flush the write buffer followed by a buffered states update and read from buffered states for graphics. And of course, it could be ignored and you can just let the engine deal with it in the next frame.

The only real difficulty here is with internal time stepping. Whenever physics-affecting logic runs that forces position/orientation to change discontinuously, the interpolation component of internal time stepping is essentially overwritten. When the interpolation gets overwritten, it's no longer consistent with the rest of the simulation, which causes a variety of 'jerky' annoyances. This is sort of an extension of the rule of thumb "use velocities to move entities instead of teleporting them, except when you're trying to teleport them."

If you're using IsFixedTimeStep or something like it and you don't feel the need to use internal time stepping, you can pretty much do whatever you want with internal properties whenever you want.
bryanedds
Posts: 14
Joined: Tue Jul 13, 2010 6:17 pm

Re: Bug: centerOfMassOffset translates object's centerPosition

Post by bryanedds »

Using internal properties, non-async updates, fixed time steps, and the following update algorithm -

Code: Select all

        protected override void Update(GameTime gameTime)
        {
            ...
            Logic(gameTime); // game logic
            Physics(gameTime); // physics
            Visualize(gameTime); // synchronize visuals to physics
            ...
        }
- the tires still draw as if drifting off.

The only thing that seems to fix it is to do visual synchronization from the _buffered_ properties instead of the internal properties.

Is that the right approach?
Last edited by bryanedds on Fri Jul 30, 2010 2:12 am, edited 1 time in total.
Norbo
Site Admin
Posts: 4929
Joined: Tue Jul 04, 2006 4:45 am

Re: Bug: centerOfMassOffset translates object's centerPosition

Post by Norbo »

Sorry if I didn't make that part clear; yes, if internal time stepping is enabled, the buffered properties and thus the vehicle wheels' world transforms will include an interpolation component. If these interpolated graphics are drawn with a car body graphically positioned using internal properties, there will appear to be drift. The car body is chronologically ahead of the wheels.
Post Reply