nichecompass.modules.compute_kl_reg_loss

nichecompass.modules.compute_kl_reg_loss(mu, logstd)

Compute Kullback-Leibler divergence as per Kingma, D. P. & Welling, M. Auto-Encoding Variational Bayes. arXiv [stat.ML] (2013). Equation (10). This will encourage encodings to distribute evenly around the center of a continuous and complete latent space, producing similar (for points close in latent space) and meaningful content after decoding.

For detailed derivation, see https://stats.stackexchange.com/questions/318748/deriving-the-kl-divergence-loss-for-vaes.

Parameters:

• mu (Tensor) – Expected values of the normal latent distribution of each node (dim: n_nodes_current_batch, n_gps).

• logstd (Tensor) – Log standard deviations of the normal latent distribution of each node (dim: n_nodes_current_batch, n_gps).

Return type:

Tensor

Returns:

kl_reg_loss: Kullback-Leibler divergence.