pymomentum.solver

Inverse kinematics and other optimizations for momentum models.

class pymomentum.solver.ErrorFunctionType

Bases: pybind11_object

Types of error functions passed to solveBodyIKProblem().

Members:

Position

Orientation

Limit

Collision

PosePrior

Motion

Projection

Distance

Vertex

Collision = <ErrorFunctionType.Collision: 3>

Distance = <ErrorFunctionType.Distance: 7>

Limit = <ErrorFunctionType.Limit: 2>

Motion = <ErrorFunctionType.Motion: 5>

Orientation = <ErrorFunctionType.Orientation: 1>

PosePrior = <ErrorFunctionType.PosePrior: 4>

Position = <ErrorFunctionType.Position: 0>

Projection = <ErrorFunctionType.Projection: 6>

Vertex = <ErrorFunctionType.Vertex: 8>

property name

property value

class pymomentum.solver.LinearSolverType

Bases: pybind11_object

Types of linear solver supported by solveBodyIKProblem().

Members:

Cholesky

QR

TrustRegionQR

Cholesky = <LinearSolverType.Cholesky: 0>

QR = <LinearSolverType.QR: 1>

TrustRegionQR = <LinearSolverType.TrustRegionQR: 2>

property name

property value

class pymomentum.solver.SolverOptions

Bases: pybind11_object

Options to control the behavior of the Levenberg-Marquardt IK solver. For advanced use cases only; most users should stick with the defaults.

property levmar_lambda: Lambda value used to regularize the Levenberg-Marquardt solver (default: 0.01f).

property line_search: Whether or not to use a line search; note that only the QR solver actually uses this (default: true).

property linear_solver: Specify the type of linear solver to use (default: QR)

property max_iter: Maximum number of nonlinear iterations (default: 50).

property min_iter: Minimum number of nonlinear iterations (default 4).

property threshold: The solver stops taking steps when the difference between iterations is less than threshold*FLT_EPS (default: 10.0f).

class pymomentum.solver.VertexConstraintType

Bases: pybind11_object

Types of error functions passed to solveBodyIKProblem().

Members:

Position

Plane

Normal

SymmetricNormal

Normal = <VertexConstraintType.Normal: 2>

Plane = <VertexConstraintType.Plane: 1>

Position = <VertexConstraintType.Position: 0>

SymmetricNormal = <VertexConstraintType.SymmetricNormal: 3>

property name

property value

pymomentum.solver.get_gradient_statistics() → tuple[int, int]

Return some basic statistics about how many IK gradients were nonzero in the backward pass of solve_ik().

This can be useful for verifying that the IK solver is converging properly; if the gradient is nonzero at the solution, the derivatives will not be computed.

Returns:: a pair [nNonZeroGradients, nTotalGradients].

pymomentum.solver.get_solve_ik_statistics() → tuple[int, int]

Return some basic statistics about ik solver in the forward pass of solve_ik().

This can be useful for detecting whether the input is appropriate; if the solve takes too many iterations, then the input is likely not in a good shape.

Returns:: a pair [nTotalSolveIK, nTotalSolveIKIterations].

pymomentum.solver.gradient(character: object, model_parameters: torch.Tensor, active_error_functions: list[pymomentum.solver.ErrorFunctionType], error_function_weights: torch.Tensor, position_cons_parents: Optional[torch.Tensor] = None, position_cons_offsets: Optional[torch.Tensor] = None, position_cons_weights: Optional[torch.Tensor] = None, position_cons_targets: Optional[torch.Tensor] = None, orientation_cons_parents: Optional[torch.Tensor] = None, orientation_cons_offsets: Optional[torch.Tensor] = None, orientation_cons_weights: Optional[torch.Tensor] = None, orientation_cons_targets: Optional[torch.Tensor] = None, pose_prior_model: object = None, motion_targets: Optional[torch.Tensor] = None, motion_weights: Optional[torch.Tensor] = None, projection_cons_projections: Optional[torch.Tensor] = None, projection_cons_parents: Optional[torch.Tensor] = None, projection_cons_offsets: Optional[torch.Tensor] = None, projection_cons_weights: Optional[torch.Tensor] = None, projection_cons_targets: Optional[torch.Tensor] = None, distance_cons_origins: Optional[torch.Tensor] = None, distance_cons_parents: Optional[torch.Tensor] = None, distance_cons_offsets: Optional[torch.Tensor] = None, distance_cons_weights: Optional[torch.Tensor] = None, distance_cons_targets: Optional[torch.Tensor] = None, vertex_cons_vertices: Optional[torch.Tensor] = None, vertex_cons_weights: Optional[torch.Tensor] = None, vertex_cons_target_positions: Optional[torch.Tensor] = None, vertex_cons_target_normals: Optional[torch.Tensor] = None, vertex_cons_type: pymomentum.solver.VertexConstraintType = <VertexConstraintType.Position: 0>) → torch.Tensor

Compute the gradient of the IK energy. .. math:

\frac{dE(\theta)}{d\theta} = 2 \sum_i \frac{dr_i(\theta)}{d\theta} r_i(\theta) = \mathbf{J}^T \mathbf{r}

We can compute gradients of the gradient() function with respect to all of the inputs.

For details on the arguments, see solve_ik().

Returns:: A (nBatch x m) torch.Tensor containing the residual.

pymomentum.solver.jacobian(character: object, model_parameters: torch.Tensor, active_error_functions: list[pymomentum.solver.ErrorFunctionType], error_function_weights: torch.Tensor, position_cons_parents: Optional[torch.Tensor] = None, position_cons_offsets: Optional[torch.Tensor] = None, position_cons_weights: Optional[torch.Tensor] = None, position_cons_targets: Optional[torch.Tensor] = None, orientation_cons_parents: Optional[torch.Tensor] = None, orientation_cons_offsets: Optional[torch.Tensor] = None, orientation_cons_weights: Optional[torch.Tensor] = None, orientation_cons_targets: Optional[torch.Tensor] = None, pose_prior_model: object = None, motion_targets: Optional[torch.Tensor] = None, motion_weights: Optional[torch.Tensor] = None, projection_cons_projections: Optional[torch.Tensor] = None, projection_cons_parents: Optional[torch.Tensor] = None, projection_cons_offsets: Optional[torch.Tensor] = None, projection_cons_weights: Optional[torch.Tensor] = None, projection_cons_targets: Optional[torch.Tensor] = None, distance_cons_origins: Optional[torch.Tensor] = None, distance_cons_parents: Optional[torch.Tensor] = None, distance_cons_offsets: Optional[torch.Tensor] = None, distance_cons_weights: Optional[torch.Tensor] = None, distance_cons_targets: Optional[torch.Tensor] = None, vertex_cons_vertices: Optional[torch.Tensor] = None, vertex_cons_weights: Optional[torch.Tensor] = None, vertex_cons_target_positions: Optional[torch.Tensor] = None, vertex_cons_target_normals: Optional[torch.Tensor] = None, vertex_cons_type: pymomentum.solver.VertexConstraintType = <VertexConstraintType.Position: 0>) → tuple[torch.Tensor, torch.Tensor]

Compute the residual and Jacobian of a body IK problem. For details on the constraint arguments, see solve_ik().

Note that some momentum error functions may reorder the constraints in a nondeterministic manner so repeated calls to this function could in principle return different results. This will not affect the gradient J^T * r, but you should avoid evaluating the residual and Jacobian in separate function calls (which is why both are returned together).

For details on the arguments, see solve_ik().

Returns:: A pair [r, J] where r is the (nBatch x m) residual and J is the (nBatch x m x nParams) Jacobian.

pymomentum.solver.reset_gradient_statistics() → None: Reset the counters used by get_gradient_statistics().

pymomentum.solver.reset_solve_ik_statistics() → None: Reset the counters used by getSolveIKStatistics().

pymomentum.solver.residual(character: object, model_parameters: torch.Tensor, active_error_functions: list[pymomentum.solver.ErrorFunctionType], error_function_weights: torch.Tensor, position_cons_parents: Optional[torch.Tensor] = None, position_cons_offsets: Optional[torch.Tensor] = None, position_cons_weights: Optional[torch.Tensor] = None, position_cons_targets: Optional[torch.Tensor] = None, orientation_cons_parents: Optional[torch.Tensor] = None, orientation_cons_offsets: Optional[torch.Tensor] = None, orientation_cons_weights: Optional[torch.Tensor] = None, orientation_cons_targets: Optional[torch.Tensor] = None, pose_prior_model: object = None, motion_targets: Optional[torch.Tensor] = None, motion_weights: Optional[torch.Tensor] = None, projection_cons_projections: Optional[torch.Tensor] = None, projection_cons_parents: Optional[torch.Tensor] = None, projection_cons_offsets: Optional[torch.Tensor] = None, projection_cons_weights: Optional[torch.Tensor] = None, projection_cons_targets: Optional[torch.Tensor] = None, distance_cons_origins: Optional[torch.Tensor] = None, distance_cons_parents: Optional[torch.Tensor] = None, distance_cons_offsets: Optional[torch.Tensor] = None, distance_cons_weights: Optional[torch.Tensor] = None, distance_cons_targets: Optional[torch.Tensor] = None, vertex_cons_vertices: Optional[torch.Tensor] = None, vertex_cons_weights: Optional[torch.Tensor] = None, vertex_cons_target_positions: Optional[torch.Tensor] = None, vertex_cons_target_normals: Optional[torch.Tensor] = None, vertex_cons_type: pymomentum.solver.VertexConstraintType = <VertexConstraintType.Position: 0>) → torch.Tensor

Compute the residual of an IK problem. The IK energy minimized by solve_ik() is the squared norm of the residual.

Note that we can only compute gradients of the residual with respect to the input model parameters.

For details on the arguments, see solve_ik().

Returns:: A (nBatch x m) torch.Tensor containing the residual.

pymomentum.solver.set_num_threads(num_threads: int) → None

Set the maximum number of threads used to solve IK problems. This is particularly useful in limiting parallelism when you have multiple pytorch processes running on a single machine and want to limit how many threads each uses to avoid CPU contention.

Parameters:: num_threads – Number of threads to use. Pass -1 to mean “number of processors”.

pymomentum.solver.solve_ik(character: object, active_parameters: torch.Tensor, model_parameters_init: torch.Tensor, active_error_functions: list[pymomentum.solver.ErrorFunctionType], error_function_weights: torch.Tensor, options: pymomentum.solver.SolverOptions = SolverOptions(linear_solver=LinearSolverType.QR, levmar_lambda=0.01, min_iter=4, max_iter=50, threshold=10, line_search=True), position_cons_parents: Optional[torch.Tensor] = None, position_cons_offsets: Optional[torch.Tensor] = None, position_cons_weights: Optional[torch.Tensor] = None, position_cons_targets: Optional[torch.Tensor] = None, orientation_cons_parents: Optional[torch.Tensor] = None, orientation_cons_offsets: Optional[torch.Tensor] = None, orientation_cons_weights: Optional[torch.Tensor] = None, orientation_cons_targets: Optional[torch.Tensor] = None, pose_prior_model: object = None, motion_targets: Optional[torch.Tensor] = None, motion_weights: Optional[torch.Tensor] = None, projection_cons_projections: Optional[torch.Tensor] = None, projection_cons_parents: Optional[torch.Tensor] = None, projection_cons_offsets: Optional[torch.Tensor] = None, projection_cons_weights: Optional[torch.Tensor] = None, projection_cons_targets: Optional[torch.Tensor] = None, distance_cons_origins: Optional[torch.Tensor] = None, distance_cons_parents: Optional[torch.Tensor] = None, distance_cons_offsets: Optional[torch.Tensor] = None, distance_cons_weights: Optional[torch.Tensor] = None, distance_cons_targets: Optional[torch.Tensor] = None, vertex_cons_vertices: Optional[torch.Tensor] = None, vertex_cons_weights: Optional[torch.Tensor] = None, vertex_cons_target_positions: Optional[torch.Tensor] = None, vertex_cons_target_normals: Optional[torch.Tensor] = None, vertex_cons_type: pymomentum.solver.VertexConstraintType = <VertexConstraintType.Position: 0>) → torch.Tensor

Batch solve a body IK problem. The IK problem is posed as a least squares problem and minimized with respect to the model parameters $θ$ using Levenberg-Marquardt. The result is differentiable with respect to most inputs.

The IK energy function that is minimized is a sum of squared residual terms, $E (θ) = \sum_{i} | | r_{i} (θ) | |_{2}^{2}$ .

We strongly recommend you use named parameters when calling this function as the parameter list is likely to grow over time.

Most tensor arguments can be specified either batched (e.g. [nBatch x nConstraints]) or unbatched (e.g. [nConstraints]). “Unbatched” tensors will be automatically expanded to match the batch dimension. So, for example, you could pass a tensor of size [nConstraints] for the positionConstraints_parents parameter and a tensor of dimension [nBatch x nConstraints x 3] for the position_constraints_targets tensor, and the solver will understand that you mean to use the same parents across the batch.

All quaternion parameters use the quaternion order [x, y, z, w], where $q = w + x i + y j + z k$ .

Returns:

The model parameters that (locally) minimize the error function, as a (nBatch x nModelParams)-dimension torch.Tensor.

Parameters:

character – Character or list of nBatch Characters to use in the solve.
active_parameters – boolean-valued torch.Tensor with dimension (k), which selects the parameters to be active during the solve. For example, you might choose to exclude scaling parameters with ‘activeParameters=~character.getScalingParameters()’. Limiting the parameters can also be helpful when dealing with underdetermined problems.
model_parameters_init – float-valued torch.Tensor which contain the (nBatch x nModelParameters)-dimensional model parameters used to initialize the solve. This could be important as the solver always converges to a local minimum.
active_error_functions – list of pymomentum.ErrorFunctionType which gives the order of input error function types.
error_function_weights – float-valued torch.Tensor with dimension (nBatch x len(activeErrorFunctions)) which contains a global weight for each active error function. The order of the weights is specified by activeErrorFunctions.
options.linearSolverType – Linear solver used in the Levenberg-Marquardt solve (default: QR).
options.levmar_lambda – Lambda value used to regularize the Levenberg-Marquardt solver (default: 0.01).
options.min_iter – Minimum number of iterations used in the nonlinear Levenberg-Marquardt solve (default: 4).
options.max_iter – Maximum number of iterations used in the nonlinear Levenberg-Marquardt solve (default: 50).
options.threshold – The solver stops taking steps when the difference between iterations is less than threshold*FLT_EPS.
options.line_search – whether to use line search for the QR solver (default: true).
position_cons_parents – integer-valued torch.Tensor of dimension (nBatch x nConstraints); contains a single parent for each constraint.
position_cons_offsets – float-valued torch.Tensor of dimension (nBatch x nConstraints x 3); contains the local offset for each constraint in its parent frame.
position_cons_weights – float-valued torch.Tensor of dimension (nBatch x nConstraints); contains a per-constraint weight.
position_cons_targets – float-valued torch.Tensor of dimension (nBatch x nConstraints x 3); contains the world-space target for each position constraint.
orientation_cons_parents – integer-valued tensor of dimension (nBatch x nConstraints); contains a single parents for each of the orientation constraints.
orientation_cons_offsets – Optional float-valued tensor of dimension (nBatch x nConstraints x 4); contains the quaternion orientation offset for each orientation constraint in its parent frame.
orientation_cons_weights – float-valued torch.Tensor of dimension (nBatch x nConstraints) which contains a per-constraint weight.
orientation_cons_targets – float-valued torch.Tensor of dimension (nBatch x nConstraints x 4) (m x n x 4): Per-constraint world-space orientation targets as quaternions.
pose_prior_model – Mixture-PCA model used by the pose prior. It can either be a pymomentum.geometry.Mpcca, or it can be a tuple of tensors [pi, mu, W, sigma, param_indices] as described in $p y m o m e n t u m . g e o m e t r y . M p p c a . t o_{t} e n s o r s$ . The former is a bit more efficient during the solve (because there is no need to re-calculate the inverse covariance matrix) but the latter case allows for differentiability.
motion_targets – float-valued torch.Tensor of dimension (nBatch x nModelParams) which contains the model parameter targets for the motionError.
motion_weights – float-valued torch.Tensor of dimension (nBatch x nModelParams) which contain a per-model-parameter weight for the motionError.
projection_cons_projections – float-valued torch.Tensor of dimension (nBatch x nConstraints x 4 x 3) containing a 4x3 projection matrix for each constraint. Note that while you can use a standard pinhole model matrix as the projection matrix, we actually recommend constructing a separate local projection matrix for each constraint centered around the camera ray, which is more robust for e.g. fisheye cameras.
projection_cons_parents – integer-valued torch.Tensor of dimension (nBatch x nConstraints); contains a single parent for each projection constraint.
projection_cons_offsets – float-valued torch.Tensor of dimension (nBatch x nConstraints x 3); contains the local offset for each projection constraint in its parent frame.
projection_cons_weights – float-valued torch.Tensor of dimension (nBatch x nConstraints); contains a per-projection constraint weight.
projection_cons_targets – float-valued torch.Tensor of dimension (nBatch x nConstraints x 2); contains the world-space target for each projection constraint.
distance_cons_origins – float-valued torch.Tensor of dimension (nBatch x nConstraints x 3); contains the world-space origins to measure distance from.
distance_cons_parents – integer-valued torch.Tensor of dimension (nBatch x nConstraints); contains a single parent for each distance constraint.
distance_cons_offsets – float-valued torch.Tensor of dimension (nBatch x nConstraints x 3); contains the local offset for each distance constraint in its parent frame.
distance_cons_weights – float-valued torch.Tensor of dimension (nBatch x nConstraints); contains a per-projection constraint weight.
distance_cons_targets – float-valued torch.Tensor of dimension (nBatch x nConstraints); contains the distance target for each constraint.
vertex_cons_vertices – int-valued torch.Tensor of dimension (nBatch x nConstraints); contains the vertex index for each constraint.
vertex_cons_weights – float-valued torch.Tensor of dimension (nBatch x nConstraints); contains the per-constraint weight for each constraint.
vertex_cons_target_positions – float-valued torch.Tensor of dimension (nBatch x nConstraints x 3); contains the target position.
vertex_cons_target_normals – float-valued torch.Tensor of dimension (nBatch x nConstraints x 3); contains the target normal. Not used if the vertex constraint type is POSITION.
vertex_cons_type – Type of vertex constraint. POSITION is a position constraint, while PLANE, NORMAL, and SYMMETRIC_NORMAL are variants on point-to-plane (PLANE uses the target normal, NORMAL uses the source normal, and SYMMETRIC_NORMAL uses a blend of the two normals).

pymomentum.solver.solve_sequence_ik(character: object, active_parameters: torch.Tensor, shared_parameters: torch.Tensor, model_parameters_init: torch.Tensor, active_error_functions: list[pymomentum.solver.ErrorFunctionType], error_function_weights: torch.Tensor, options: pymomentum.solver.SolverOptions = SolverOptions(linear_solver=LinearSolverType.QR, levmar_lambda=0.01, min_iter=4, max_iter=50, threshold=10, line_search=True), position_cons_parents: Optional[torch.Tensor] = None, position_cons_offsets: Optional[torch.Tensor] = None, position_cons_weights: Optional[torch.Tensor] = None, position_cons_targets: Optional[torch.Tensor] = None, orientation_cons_parents: Optional[torch.Tensor] = None, orientation_cons_offsets: Optional[torch.Tensor] = None, orientation_cons_weights: Optional[torch.Tensor] = None, orientation_cons_targets: Optional[torch.Tensor] = None, pose_prior_model: object = None, motion_targets: Optional[torch.Tensor] = None, motion_weights: Optional[torch.Tensor] = None, projection_cons_projections: Optional[torch.Tensor] = None, projection_cons_parents: Optional[torch.Tensor] = None, projection_cons_offsets: Optional[torch.Tensor] = None, projection_cons_weights: Optional[torch.Tensor] = None, projection_cons_targets: Optional[torch.Tensor] = None, distance_cons_origins: Optional[torch.Tensor] = None, distance_cons_parents: Optional[torch.Tensor] = None, distance_cons_offsets: Optional[torch.Tensor] = None, distance_cons_weights: Optional[torch.Tensor] = None, distance_cons_targets: Optional[torch.Tensor] = None, vertex_cons_vertices: Optional[torch.Tensor] = None, vertex_cons_weights: Optional[torch.Tensor] = None, vertex_cons_target_positions: Optional[torch.Tensor] = None, vertex_cons_target_normals: Optional[torch.Tensor] = None, vertex_cons_type: pymomentum.solver.VertexConstraintType = <VertexConstraintType.Position: 0>) → torch.Tensor

Batch solve a body multi-frame, or sequence, IK problem, with some parameters unique to each frame and some parameters shared between frames. A typical use case is to solve for a single body scale plus a per-frame pose given per-frame constraints.

The constraints are the same ones available in solve_ik(), except that instead of having tensor dimensions [nBatch x …] they use tensor dimension [nBatch x nFrames x …]. For example: posConstraint_parents would be [nBatch x nFrames x nConstraints]. Note that you can choose to elide both the per-batch and per-frame dimension (if you want to share parents across all frames and the entire batch) but you cannot skip one or the other unless nBatch is 1. This is to prevent confusing ambiguities in whether you meant sharing across the batch dimension or the frame dimension.

Parameters:

character – Character or list of nBatch Characters to use in the solve.
activeParameters – boolean-valued torch.Tensor with dimension (k), which selects all the parameters (both shared and per-frame) to be active during the solve.
sharedParameters – boolean-valued torch.Tensor with dimension (k), which selects all the parameters to be shared across the frames (typically this would be scale or shape parameters).
modelParameters_init – float-valued torch.Tensor which contain the ([nBatch] x nFrames x nModelParameters)-dimensional model parameters used to initialize the solve. This could be important as the solver always converges to a local minimum.
activeErrorFunctions – list of pymomentum.ErrorFunctionType which gives the order of input error function types.
errorFunctionWeights – float-valued torch.Tensor with dimension (nBatch x nFrames x len(activeErrorFunctions)) which contains a per-frame global weight for each active error function. The order of the weights is specified by activeErrorFunctions.

pymomentum.solver.transform_pose(character: pymomentum.geometry.Character, model_parameters: torch.Tensor, transform: torch.Tensor, ensure_continuous_output: bool = True) → torch.Tensor

Computes a new set of model parameters such that the character pose is a rigidly transformed version of the original pose. While there is technically a closed form solution for any given skeleton, this is complicated in momentum because different characters attach the rigid parameters to different joints, so a fully general solution uses IK. This function optimizes the IK to run on a minimal set of joints and be as fast as possible, while dealing with local minima problems due to Euler angles.

Returns:

A new set of model parameters such that the skeleton’s root has been transformed.

Parameters:

character – Character to use in the solve.
model_parameters – Model parameters for the posed character in its initial space.
transform – The rigid transform to apply to the character, specified as a skeleton state(tx, ty, tz, rx, ry, rz, rw, s). Note that the scale is ignored.
ensure_continuous_output – If true, the solver will try to ensure that the output is continuous in time. This helps to remove Euler flips in continuous data.