Symbol ∇ nabla

$\nabla$ = gradient symbol

${\nabla }_{a}C$ (C = cost, a = activation)

• ${\nabla }_{a}C$ = gradient of C with respect to a
• Meaning: how the cost C changes when activation a changes
• Intuition: if a changes a little, how much will the cost change?
  • ∇ₐC > 0, increasing a a little will increase the cost
  • ∇ₐC < 0, increasing a a little will decrease the cost
  • It tells us the direction and strength of how activation affects cost