KISS: Hand crafted JSON is NOT faster than ObjectMapper

While going through some code earlier today, I encountered a method that attempted to escape quotes and back slashes in a poor way. The author of that method presumably thought that it’d be way faster than Jackson’s ObjectMapper.

Here’s what they wrote:

public static String escapeString(Object o) {
if (o == null) {
return null;
}
String str = o.toString();
if (str.contains("\\")) {
str = str.replace("\\", "\\\\");
}
if (str.contains("\"")) {
str = str.replace("\"", "\\\"");
}
return str;
}
view raw A.java hosted with ❤ by GitHub

This produced illegal JSON strings, especially when the input string had new lines and carriage returns in it.

Here’s what I replaced it with:

private static final ObjectMapper OM = new ObjectMapper();
public static String escapeString(Object o) {
if (o == null) {
return null;
}
try {
// The string is automatically quoted. Therefore, return a new string that
// doesn't contain those quotes (since the caller appends quotes themselves).
val bytes = OM.writeValueAsBytes(o.toString());
return new String(bytes, 1, bytes.length2, StandardCharsets.UTF_8);
} catch (JsonProcessingException e) {
return "";
}
}
view raw A.java hosted with ❤ by GitHub

At first, I was uncertain about the efficiency of either approaches. Let’s JMH it:

private static final ObjectMapper OM = new ObjectMapper();
private static final String STR = "hello world – the quick brown fox jumps over "
+ "the lazy dog\r\n\r\nand here's "
+ "a random slash\\, and some \"s";
@Benchmark
public void jsonStringSerialization(final Blackhole blackhole) throws Exception {
byte[] obj = OM.writeValueAsBytes(STR);
blackhole.consume(new String(obj, 1, obj.length2, StandardCharsets.UTF_8));
}
@Benchmark
public void jsonStringManual(final Blackhole blackhole) {
String str = STR;
if (str.contains("\\")) {
str = str.replace("\\", "\\\\");
}
if (str.contains("\"")) {
str = str.replace("\"", "\\\"");
}
blackhole.consume(str);
}
view raw A.java hosted with ❤ by GitHub

The results were quite astounding. I hadn’t expected something like the following:

Benchmark                            Mode  Cnt        Score   Error  Units
Benchmarks.jsonStringManual         thrpt    2    83301.447          ops/s
Benchmarks.jsonStringSerialization  thrpt    2  4171309.830          ops/s

There must be something wrong, right? Perhaps it’s because of the static string. Let’s replace our static string with a random one generated for each iteration:

private static final ObjectMapper OM = new ObjectMapper();
@Benchmark
public void jsonStringSerialization(final Blackhole blackhole) throws Exception {
byte[] obj = OM.writeValueAsBytes(randomString());
blackhole.consume(new String(obj, 1, obj.length2, StandardCharsets.UTF_8));
}
@Benchmark
public void jsonStringManual(final Blackhole blackhole) {
String str = randomString();
if (str.contains("\\")) {
str = str.replace("\\", "\\\\");
}
if (str.contains("\"")) {
str = str.replace("\"", "\\\"");
}
blackhole.consume(str);
}
private static String randomString() {
return RandomStringUtils.random(75,
'a', 'b', 'c', 'd', 'e', 'f', 'g', 'h', 'i', 'j', 'k', 'l',
'm', 'n', 'o', 'p', 'q', 'r', 's', 't', 'u', 'v', 'w', 'x',
'y', 'z',
'\\', '\r', '\n', '\"', '\'', ' ');
}
@Benchmark
public void randomString(final Blackhole blackhole) {
blackhole.consume(randomString());
}
view raw A.java hosted with ❤ by GitHub

Here’s the JMH report:

Benchmark                            Mode  Cnt        Score   Error  Units
Benchmarks.jsonStringManual         thrpt    2   133432.951          ops/s
Benchmarks.jsonStringSerialization  thrpt    2  1535802.541          ops/s
Benchmarks.randomString             thrpt    2  2871443.990          ops/s

Conclusion

KISS. Premature optimisations are harmful. Not only can they introduce bugs, but they could be slower than the industry standard. Benchmark everything carefully.

Taming a throttled API with Dynamic Proxies in Java

Recently, at CleverTap, we’ve begun migrating some of our largest clusters to a new protocol (for starters, think ~115 instances at a time). One of the most fun things I’ve had my hands on during this migration was the AWS Systems Manager API.

When we scaled up our migrations gradually from a 10 node cluster, we were challenged with dealing with API throttling exceptions (because sure, who wouldn’t throttle their APIs?). There were two immediate solutions that hit our mind:

  1. Review every usage of the SSM client and handle the throttling exception gracefully
  2. Wrap the SSM client and handle the throttling exception transparently

Naturally, we settled for option 2. I am a big fan of hidden abstractions. So what did we do? We implemented the AWS interface in question, only to discover that we’d have to handle a ton of methods individually (obviously copy/paste). There had to be a better solution!

And then, Google did it’s thing. We discovered Dynamic Proxies. And viola! We were able to transparently handle and implement an auto retry strategy within just 14 lines!

Here’s what it looked like:

MyStubbornAPIInterface actualInstance = … // Create it however you'd create your original instance.
MyStubbornAPIInterface proxiedInstance = (MyStubbornAPIInterface) Proxy.newProxyInstance(actualInstance.getClass().getClassLoader(),
new Class[]{MyStubbornAPIInterface.class}, (proxy, method, args) -> {
while (true) {
try {
return method.invoke(actualInstance, args);
} catch (MyThrottlingException e) {
try {
Thread.sleep(ThreadLocalRandom.current().nextInt(1, 5) * 1000L);
} catch (InterruptedException e) {
Thread.currentThread().interrupt();
}
}
}
});

The code above can be easily adapted to various SDKs (in our case, it was the AWS SDK).

Now, all we had to do was pass around this proxied instance, and viola, the consumers of this API had no clue that the API implemented an auto retry mechanism!

Cheers!

IntelliJ on steroids with G1 GC

Lately, I noticed that IntelliJ started to pause for quite some time during it’s GC cycles, and that it was very frequent when I was editing three files (over 1.2k LOC each) split vertically.

The current version of IntelliJ runs on a bundled version of Java 1.8, who’s default garbage collector is Parallel GC. While this works for most people, it didn’t for me.

After a ton of reading up on how GC works, and the fine tuning parameters for G1, I put it to use. Here’s a copy of my idea.vmoptions file:


-Xmx2048m
-Xss256k
-XX:+UseG1GC
-XX:InitiatingHeapOccupancyPercent=65
-XX:G1HeapRegionSize=16m
-XX:MaxGCPauseMillis=100

view raw

idea.vmoptions

hosted with ❤ by GitHub

There was an instant performance boost in the IDE – it was far more responsive than ever before. The pauses have disappeared, and it’s super snappy :)

Note: As a general rule of thumb, don’t increase the maximum memory allocated to the IDE beyond 2 gigabytes – it’s just not worth it.

0f > Float.MIN_VALUE = false!

Updates:
Turns out that this is the expected behaviour from the java doc:
A constant holding the smallest positive nonzero value of type float, 2-149.

So now, how do I get the smallest negative value that a float can hold?


Just came across the most weirdest thing ever in Java – 0f > Float.MIN_VALUE returns false!

Similarly, anything less than 0, (say -10 for example) is surprisingly not greater than Float.MIN_VALUE.

However, 0 > Integer.MIN_VALUE returns true!

Want to try it out?

public class E {
    public static void main(String[] args) {
        System.out.println("1f > Float.MIN_VALUE: " + (1f > Float.MIN_VALUE));
        System.out.println("0f > Float.MIN_VALUE: " + (0f > Float.MIN_VALUE));
        System.out.println("-1f > Float.MIN_VALUE: " + (-1f > Float.MIN_VALUE));
        System.out.println("0f < Float.MIN_VALUE: " + (0f < Float.MIN_VALUE));
        System.out.println("0f == Float.MIN_VALUE: " + (0f == Float.MIN_VALUE));
    }
}

Result:

1f > Float.MIN_VALUE: true
0f > Float.MIN_VALUE: false
-1f > Float.MIN_VALUE: false
0f < Float.MIN_VALUE: true
0f == Float.MIN_VALUE: false

How to get a File object of an Android raw resource using reflection

Context:

You have a URI to a resource which is placed in the raw directory inside the Android resources directory, res. Say you want to take that raw resource(let’s say it’s an image) and add it to an attachment using FIleBody to a MultiPartEntity. Both these classes are available in the Apache HTTP Components library. FileBody will only allow you to give it a File object in it’s constructor. Certainly, you are stuck as you have the URI to an Android raw resource, and CANNOT create a File object out of it.

 

So what’s the solution? Reflection.

    private File getFileFromRawResource(Uri rUri) {
        String uri = rUri.toString();
        String fn;
        // I've only tested this with raw resources
        if (uri.contains("/raw/")) {
            // Try to get the resource name
            String[] parts = uri.split("/");
            fn = parts[parts.length - 1];
        } else {
            return null;
        }
        // Notice that I've hard-coded the file extension to .jpg
        // I was working with getting a File object of a JPEG image from my raw resources
        String dest = Environment.getExternalStorageDirectory() + "/image.jpg";
        try {
            // Use reflection to get resource ID of the raw resource
            // as we need to get an InputStream to it
            // getResources(),openRawResource() takes only a resource ID
            R.raw r = new R.raw();
            Field frame = R.raw.class.getDeclaredField(fn);
            frame.setAccessible(true);
            int id = (Integer) frame.get(r);
            // Get the InputStream
            InputStream inputStream = getResources().openRawResource(id);
            FileOutputStream fileOutputStream = new FileOutputStream(dest);
            // IOUtils is a class from Apache Commons IO
            // It writes an InputStream to an OutputStream
            IOUtils.copy(inputStream, fileOutputStream);
            fileOutputStream.close();
            return new File(dest);
        } catch (NoSuchFieldException e) {
            Log.e("MyApp", "NoSuchFieldException in getFileFromRawResource()");
        } catch (IllegalAccessException e) {
            Log.e("MyApp", "IllegalAccessException in getFileFromRawResource()");
        } catch (FileNotFoundException e) {
            Log.e("MyApp", "FileNotFoundException in getFileFromRawResource()");
        } catch (IOException e) {
            Log.e("MyApp", "IOException in getFileFromRawResource()");
        }
        return null;
    }    

The code is pretty much self-explanatory. Drop a comment if you have any questions/doubts.