Numerically stable binary log loss

Question

Implement mean binary cross-entropy directly from finite logits and labels in {0, 1}, avoiding overflow for logits near plus or minus 1000.

Short answer

For logit z and label y, use max(z,0) - z*y + log(1 + exp(-abs(z))). The exponential argument is non-positive. Validate shapes and labels instead of silently broadcasting.

Original question and answer written by SelfMadeQuant.com.

See the hint and the full worked solution in the question bank (free account).

Open the worked solution

Lessons behind this question

The Supervised Learning Problem and the ML Pipeline

Start with a free lesson: Basic Statistics and Regression is part of a free topic. Create a free account, no card needed, to read it in full with worked solutions.

Read it free