Once you accept that each float represent a range of reals most design decisions make sense.
Infinity is just the name given to values that are too large to represent otherwise and negative zero includes negative values too close to zero.

What mathematics probably hate most about floats is that rounding breaks commutativity.