Introduction

In deep learning, predicting properties of objects or scenes in a 360-degree view has emerged as a crucial challenge. This task is especially relevant in fields like virtual reality, autonomous driving, and robotics, where models must learn and predict spherical data. The Keras library offers various ways to implement custom loss functions tailored to 360-degree prediction tasks.

The core difficulty is that spherical data wraps around, so a prediction at 359 degrees is actually very close to one at 1 degree. Standard loss functions like MSE do not understand this wrap-around property, which leads to enormous penalties for predictions that are geometrically close but numerically far apart.

Understanding 360-Degree Data

360-degree data inherently differs from traditional planar data because of its spherical nature. This unique characteristic necessitates predictions that respect continuity and wrap-around properties of spherical surfaces.

• Continuity and Seamlessness: The left-most edge seamlessly connects to the right-most edge. An angle of 0 degrees is identical to 360 degrees.
• Distortion Consideration: Tangent plane projections, such as the equirectangular format, can distort area proportions, affecting predictions at different latitudes.
• Non-Euclidean Geometry: Distances on a sphere follow great-circle arcs, not straight lines. Two points that appear far apart in a flat projection may be close on the actual sphere.

Loss Functions for 360-Degree Predictions

When dealing with 360-degree predictions, conventional loss functions like Mean Squared Error (MSE) need modification to account for the data's spherical properties. Central requirements for a suitable loss function include:

• Spherical Distance Measurement: The loss function must measure the angular difference between predictions and targets effectively.
• Handling Wrap-Around: Seamless edge handling should be inherent to the function.

Angular MSE Loss

The simplest approach for angular predictions is to compute the angular difference modulo $2\pi$ (or 360 degrees) and then apply MSE. For a predicted angle $\hat{\theta}$ and a target angle $\theta$, the angular distance is:

$$d(\hat{\theta}, \theta) = \min(|\hat{\theta} - \theta|, 2\pi - |\hat{\theta} - \theta|)$$

The loss over a batch of $N$ predictions becomes:

$$L = \frac{1}{N} \sum_{i=1}^{N} d(\hat{\theta}_i, \theta_i)^2$$

Here is a Keras implementation:

1import tensorflow as tf
2from tensorflow import keras
3
4def angular_mse_loss(y_true, y_pred):
5    diff = y_true - y_pred
6    # Wrap difference into [-pi, pi]
7    diff = tf.math.atan2(tf.math.sin(diff), tf.math.cos(diff))
8    return tf.reduce_mean(tf.square(diff))

Spherical MSE (Haversine-Based)

For full spherical coordinates (latitude and longitude), this variation of MSE uses the angular distance derived from the spherical law of cosines or the Haversine formula. The spherical distance between two points $P_1(\phi_1, \lambda_1)$ and $P_2(\phi_2, \lambda_2)$ is calculated using:

$$d = \arccos\left(\sin\phi_1 \sin\phi_2 + \cos\phi_1 \cos\phi_2 \cos(\lambda_2 - \lambda_1)\right)$$

An equivalent Haversine formulation avoids numerical issues for small distances:

$$a = \sin^2\left(\frac{\Delta\phi}{2}\right) + \cos\phi_1 \cos\phi_2 \sin^2\left(\frac{\Delta\lambda}{2}\right)$$

$$d = 2 \arctan2\left(\sqrt{a}, \sqrt{1 - a}\right)$$

1def haversine_loss(y_true, y_pred):
2    phi1, lam1 = y_true[:, 0], y_true[:, 1]
3    phi2, lam2 = y_pred[:, 0], y_pred[:, 1]
4
5    dphi = phi2 - phi1
6    dlam = lam2 - lam1
7
8    a = tf.square(tf.sin(dphi / 2)) + \
9        tf.cos(phi1) * tf.cos(phi2) * tf.square(tf.sin(dlam / 2))
10    d = 2 * tf.atan2(tf.sqrt(a), tf.sqrt(1 - a))
11    return tf.reduce_mean(tf.square(d))

Cosine-Based Loss

Another effective approach represents angles as unit vectors $(\cos\theta, \sin\theta)$ and uses cosine similarity. The loss penalizes the angular difference without wrap-around issues because the representation is inherently circular:

$$L = 1 - \cos(\hat{\theta} - \theta)$$

def cosine_angle_loss(y_true, y_pred):
    return tf.reduce_mean(1.0 - tf.cos(y_true - y_pred))

Common Challenges and Solutions

• Maintaining Spherical Continuity in Flat Images: Use spherical projection formats, like equirectangular, that preserve important properties. Consider sampling strategies that adjust to different latitudes.
• Traditional CNNs Missing Spherical Correlations: Incorporate spherical convolutions or graph-based networks tailored for spherical domains.
• Standard Metrics Being Ill-Suited: Employ metrics based on geodesic distances or adapted accuracy measures that consider spherical topology.

Practical Applications

• Virtual Reality: Generating seamless 360-degree content where seam artifacts at the wrap-around boundary must be minimized.
• Autonomous Vehicles: Understanding full-surround environments ensuring safer navigation. LiDAR and camera fusion often produces spherical data.
• Robotics: Comprehensive environmental mapping for robotic decision-making, where the robot needs a consistent 360-degree world model.

Summary

Choosing the right loss function for 360-degree predictions depends on the specific task geometry. For single-angle predictions (heading, yaw), angular MSE or cosine loss works well. For full spherical coordinates, the Haversine-based loss is more appropriate. In all cases, the loss function must respect the periodic nature of angular data to produce meaningful gradients during training.