The Math.abs timebomb
Overview
In a system I worked on in the past, we had a Kafka consumer that consumed messages from a Kafka topic. The consumer consumes the message and acted as a router of sorts - its only other responsibility was to forward the messages to 3 agents (worker threads) that independently process the message.
There’s no ordering requirement - there’s no message-agent affinity required here. The distribution algorithm could just as easily be round-robin.
The implementation of the Consumer is roughly:
public class Consumer {
private final Agent[] agents;
public void consume(Message message, String key) {
int agent = Math.abs(key.hashCode()) % agents.length;
agents[agent].process(message);
}
}This code has a timebomb in it. Can you spot it?
Kaboom
On a nice and quiet afternoon, an ArrayIndexOutOfBoundsException surfaces, breaking our ingress data feed.
We have a production incident on our hands.
We scramble to figure out what went wrong, and scanning through the code, we don’t see anything wrong with it.
After some digging, someone uncovers the root cause -
The Timebomb
What happens if the key happens to be HZcxf_?
System.out.println("HZcxf_".hashCode()); // -2147483648, -2^31, or Integer.MIN_VALUEWhat should we expect when we call Math.abs(-2^31)? We should get 2^31, right?
System.out.println(Math.abs(-2147483648)); // -2147483648The reason for this is because -2147483648 is the smallest possible value that can be represented by a signed 32-bit
integer. Math.abs can’t return +2147483648 here, since it is out of the range of a signed 32-bit integer. The maximum
possible value that can be represented by a signed 32-bit integer is 2147483647, or Integer.MAX_VALUE. 2147483648
can only be represented by a long in Java.
The Fix
Option A.
By changing the code to the above, we ensure that the hashCode is first taken modulo the number of agents before
- int agent = Math.abs(key.hashCode()) % agents.length;
+ int agent = Math.abs(key.hashCode() % agents.length);Option B.
Alternatively, we can use the Math.floorMod method introduced in Java 8. Most importantly, it is guaranteed to
return a non-negative result.
- int agent = Math.abs(key.hashCode()) % agents.length;
+ int agent = Math.floorMod(key.hashCode(), agents.length);Other Languages?
The behaviour varies across languages. Typically, for languages that have unbounded integers, this scenario is less likely to occur. Languages that have fixed-width integers, such as Java and Go, are more likely to encounter this issue.
The Go standard library does not provide an Abs function for integers. The math package provides an Abs function for
float64 values, but not for ints. A common implementation of an Abs function for integers is shown below, which
can yield the same int overflow issue.
func Abs(x int) int {
if x < 0 {
return -x
}
return x
}
func main() {
// 2^63 - 1 is the largest possible value that can be represented by a signed 64-bit integer
// As expected, this Abs call returns the same negative value due to an int overflow.
fmt.Println(Abs(-9_223_372_036_854_775_808))
}TLDR
Be wary of the range of values that can be represented by the data types you’re working with. This is an extremely subtle bug that can occur in the most unexpected of places, at the worst possible times.