pose_format.utils.siren module

Classes:

PoseDataset(pose)

Dataset for pose

SineLayer(in_features, out_features[, bias, ...])

A sine activation layer as described in the SIREN paper.

Siren(in_features, hidden_features, ...[, ...])

SIREN network consisting of SineLayers.

Functions:

get_pose_siren(pose[, hidden_features, ...])

For training: returns a SIREN model for pose data

masked_mse_loss(model_output, ground_truth, ...)

For calculating masked mean squared error loss
class pose_format.utils.siren.PoseDataset(pose)[source]

Bases: Dataset

Dataset for pose

Parameters:

pose (Pose) – Pose object containing body data & confidence scores

Note

  • Assumes pose data is provided in a specific format with body data & confidence scores.

Methods:

get_coords(time, fps)

Gets coordinates based on time and frames per second (fps)
static get_coords(time, fps)[source]

Gets coordinates based on time and frames per second (fps)

Parameters:

  • time (float) – Time (duration)

  • fps (float) – frames per second

Returns:

Tensor with coordinates

Return type:

torch.Tensor

class pose_format.utils.siren.SineLayer(in_features, out_features, bias=True, is_first=False, omega_0=30)[source]

Bases: Module

A sine activation layer as described in the SIREN paper.

Parameters:

  • in_features (int) – number of input features.

  • out_features (int) – number of output features.

  • bias (bool, optional) – If set to False, the layer will not learn an additive bias. Default is True.

  • is_first (bool, optional) – If it’s the first layer in the network. Default is False.

  • omega_0 (float, optional) – hyperparameter. Default is 30.

omega_0

hyperparameter for controlling the sine function.

Type:

float

is_first

Determines how weights are initialized.

Type:

bool

Note

  • See paper sec. 3.2, final paragraph, and supplement Sec. 1.5 for discussion of omega_0.

  • If is_first=True, omega_0 multiplies the activations before the nonlinearity.

  • If is_first=False, weights are divided by omega_0 to keep magnitude of activations constant.

Methods:

forward(input)

forward pass through layer

forward_with_intermediate(input)

Forward pass with intermediate value, before sine act.

init_weights()

initializes weights

Attributes:

training

forward(input)[source]

forward pass through layer

Parameters:

input (torch.Tensor) – input tensor to layer

Returns:

Sine “activated” output tensor

Return type:

torch.Tensor

forward_with_intermediate(input)[source]

Forward pass with intermediate value, before sine act. For visualization of activation distributions

Parameters:

input (torch.Tensor) – Input tensor to layer

Returns:

Sine activated output tensor along with an intermediate tensor

Return type:

tuple

init_weights()[source]

initializes weights

training: bool
class pose_format.utils.siren.Siren(in_features, hidden_features, hidden_layers, out_features, outermost_linear=False, first_omega_0=30, hidden_omega_0=30.0)[source]

Bases: Module

SIREN network consisting of SineLayers.

Parameters:

  • in_features (int) – number of input features.

  • hidden_features (int) – number of hidden features.

  • hidden_layers (int) – number hidden layers.

  • out_features (int) – number output features.

  • outermost_linear (bool, optional) – If True, outermost layer is linear. Default- False.

  • first_omega_0 (float, optional) – Omega_0 for the first layer. The default is 30

  • hidden_omega_0 (float, optional) – Omega_0 for the hidden layers, default; 30

Methods:

forward(coords)

Forward pass through network

forward_with_activations(coords[, retain_grad])

Returns not only model output, but also intermediate activations.

Attributes:

training

forward(coords)[source]

Forward pass through network

Parameters:

coords (torch.Tensor) – Input coordinates

Returns:

output (network) and coordinates (input)

Return type:

tuple

forward_with_activations(coords, retain_grad=False)[source]

Returns not only model output, but also intermediate activations. Only used for visualizing activations later!

training: bool
pose_format.utils.siren.get_pose_siren(pose, hidden_features=256, hidden_layers=4, total_steps=5000, learning_rate=1e-05, batch_size=1, steps_til_summary=None, cuda=True)[source]

For training: returns a SIREN model for pose data

Parameters:

  • pose (Pose) – Pose object with body data & confidence

  • hidden_features (int, optional) – hidden features. Default 256.

  • hidden_layers (int, optional) – hidden layers, default 4.

  • total_steps (int, optional) – total (training) steps, default; 5000.

  • learning_rate (float, optional) – Learning rate (optimization) default; 1e-5.

  • batch_size (int, optional) – batch size (training) default; 1.

  • steps_til_summary (int, optional) – Steps until summary. If None, no summary is printed, default is None

  • cuda (bool, optional) – If True, training on GPU, default is True

Returns:

prediction function with model output of trained SIREN model

Return type:

function

pose_format.utils.siren.masked_mse_loss(model_output, ground_truth, confidence)[source]

For calculating masked mean squared error loss

Parameters:

  • model_output (torch.FloatTensor) – output of model

  • ground_truth (torch.FloatTensor) – Ground truth values

  • confidence (torch.FloatTensor) – Confidence scores for pose

Returns:

computed masked mse loss

Return type:

torch.Tensor