Taming a throttled API with Dynamic Proxies in Java

Recently, at CleverTap, we’ve begun migrating some of our largest clusters to a new protocol (for starters, think ~115 instances at a time). One of the most fun things I’ve had my hands on during this migration was the AWS Systems Manager API.

When we scaled up our migrations gradually from a 10 node cluster, we were challenged with dealing with API throttling exceptions (because sure, who wouldn’t throttle their APIs?). There were two immediate solutions that hit our mind:

  1. Review every usage of the SSM client and handle the throttling exception gracefully
  2. Wrap the SSM client and handle the throttling exception transparently

Naturally, we settled for option 2. I am a big fan of hidden abstractions. So what did we do? We implemented the AWS interface in question, only to discover that we’d have to handle a ton of methods individually (obviously copy/paste). There had to be a better solution!

And then, Google did it’s thing. We discovered Dynamic Proxies. And viola! We were able to transparently handle and implement an auto retry strategy within just 14 lines!

Here’s what it looked like:

MyStubbornAPIInterface actualInstance = // Create it however you'd create your original instance.
MyStubbornAPIInterface proxiedInstance = (MyStubbornAPIInterface) Proxy.newProxyInstance(actualInstance.getClass().getClassLoader(),
new Class[]{MyStubbornAPIInterface.class}, (proxy, method, args) > {
while (true) {
try {
return method.invoke(actualInstance, args);
} catch (MyThrottlingException e) {
try {
Thread.sleep(ThreadLocalRandom.current().nextInt(1, 5) * 1000L);
} catch (InterruptedException e) {

The code above can be easily adapted to various SDKs (in our case, it was the AWS SDK).

Now, all we had to do was pass around this proxied instance, and viola, the consumers of this API had no clue that the API implemented an auto retry mechanism!


Sending OTA updates over WiFi to your ESP8266

This Christmas, I added a whole bunch of lights powered by 5V power sources. My goal was to switch them on at sunset, and switch them off on sunrise, by using a MOSFET for power control :)

While I was doing this, I wanted to send OTA updates of my Lua files to the ESP8266 via WiFi. For some unknown reason, I couldn’t use luatool.py’s TCP update method.

So, I ended up building my very own OTA update protocol (which turned out to be fun!). To begin, add ota.lua to your project, and invoke it using dofile("ota.lua") in your init.lua:

Send OTA updates to remotely update lua scripts on your ESP8266.
LICENCE: http://opensource.org/licenses/MIT
Created by Jude Pereira <contact@judepereira.com>
See https://judepereira.com/blog/sending-ota-updates-over-wifi-to-your-esp8266/
srv = net.createServer(net.TCP)
current_file_name = nil
srv:listen(8080, function(conn)
conn:on("receive", function(sck, payload)
if string.sub(payload, 1, 5) == "BEGIN" then
current_file_name = string.sub(payload, 7)
file.open(current_file_name, "w")
sck:send("NodeMCU: Writing to " .. current_file_name .. '\n')
elseif string.sub(payload, 1, 4) == "DONE" then
sck:send("NodeMCU: Wrote file " .. current_file_name .. "!\n")
current_file_name = nil
elseif string.sub(payload, 1, 7) == "RESTART" then
sck:send("NodeMCU: Restart!\n")
tmr.create():alarm(500, tmr.ALARM_SINGLE, node.restart)
if file.open(current_file_name, "a+") then
if file.write(payload) then
sck:send("NodeMCU: Write failed!\n")
sck:send("NodeMCU: Open failed!\n")
conn:on("sent", function(sck) sck:close() end)
view raw ota.lua hosted with ❤ by GitHub

Then, to use this shiny new TCP endpoint created on your ESP8266/NodeMCU, create a wrapper shell script:

# Wrapper script for sending OTA updates to your ESP8266 running NodeMCU.
# See https://judepereira.com/blog/sending-ota-updates-over-wifi-to-your-esp8266/
for i in "$@"; do
echo "Sending $i"
echo -n "BEGIN $FILE" | nc $HOST $PORT
while read -r line; do
#echo -n "write: $line … "
if ! echo "$line" | nc $HOST $PORT | grep "ok" &>/dev/null; then
echo "Write failed! Please retry…"
exit 1
done <"$FILE"
echo -n "DONE" | nc $HOST $PORT
echo -n "RESTART" | nc $HOST $PORT
view raw ota.sh hosted with ❤ by GitHub

Heads up! Replace HOST with the IP of your NodeMCU.

The wrapper script will automatically trigger a restart at the end. To use the wrapper script:

$ chmod +x ota.sh
$ ./ota.sh file1.lua file2.lua init.lua

And that’s it! OTA update away!

nRF52840 – CircuitPython 5.0.0 pinout

I recently got CircuitPython running on my SparkFun nRF52840 Pro Mini.

Compared to what SparkFun says the pin mappings should be, I found them to be quite different. Perhaps they changed with CircuitPython 5.0.0?

Here’s what the pin mapping looks like, when superimposed over SparkFun’s pinout diagram:

SparkFun Pro nRF52840 Mini pinout with CircuitPython superimposed


nRF52840 – flashing the s340 v6.1.1 SoftDevice

This post is a work in progress (WIP). The result of this experiment is a success. I have flashed my SparkFun nRF52840 mini, and I’m able to run the bicycle combined speed & cadence sensor example.

Before we begin,  a big hats off to Charles, who brought support for the SparkFun board I have to the Adafruit nRF52 bootloader. Cheers Charles! I owe you a beer :) – GitHub profile, blog

Important software versions:

nRF SDK: nRF5_SDK_15.3.0_59ac345
ARM GCC: 8.2.1
s340: s340_nrf52_6.1.1
board: SparkFun Pro nRF52840 mini

Rough outline:

1. Checkout ‘s PR

2. Copy over src/linker/s140_v6.ld to src/linker/s340_v6.ld – there are zero differences between these two files

3. Patch your main.c from the checked out source to initialise the soft device with the ANT_LICENSE_KEY

diff –git a/src/main.c b/src/main.c
index 8ac1dba..2e43f49 100644
— a/src/main.c
+++ b/src/main.c
@@ -301,7 +301,7 @@ static uint32_t softdev_init(bool init_softdevice)
APP_ERROR_CHECK( sd_softdevice_enable(&clock_cfg, app_error_fault_handler) );
+ APP_ERROR_CHECK( sd_softdevice_enable(&clock_cfg, app_error_fault_handler, ANT_LICENSE_KEY) );
/*————- Configure BLE params ————-*/

view raw
hosted with ❤ by GitHub

4. Patch the Makefile to use the s340 soft device files

diff –git a/Makefile b/Makefile
index 6dbaf98..4acd319 100644
— a/Makefile
+++ b/Makefile
@@ -104,7 +104,7 @@ ifneq ($(IS_52832),)
SD_NAME = s132
SD_NAME = s140
+SD_NAME = s340
DFU_DEV_REV = 52840
@@ -275,7 +275,7 @@ CFLAGS += -DNRF52832_XXAA
CFLAGS += -DS132
CFLAGS += -DS140
+CFLAGS += -DS340
@@ -314,7 +314,7 @@ ASMFLAGS += -DNRF52

view raw
hosted with ❤ by GitHub

5. Place the contents of the s340 archive (sign up for the evaluation licence from thisisant.com, wait for 1 business day, and then download the s340 soft device)

$ tree lib/softdevice/s340_nrf52_6.1.1/
├── s340_nrf52_6.1.1_API
│   └── include (all header files must be under here)
└── s340_nrf52_6.1.1_softdevice.hex

view raw
hosted with ❤ by GitHub

6. Flash your nRF52840 device (double reset to enter the DFU mode)

$ make BOARD=sparkfun_pro_nrf52840_mini SERIAL=/dev/tty.usbmodem14301 dfu-flash

7. Verify

When you enter DFU mode after the above command completes, the contents of INFO_UF2.TXT must look something like the contents here:

UF2 Bootloader 0.2.10-4-g79fe6cc-dirty lib/nrfx (v1.1.0-1-g096e770) lib/tinyusb (legacy-755-g55874813) s340 6.1.1
Model: SparkFun Pro nRF52840 Mini
Board-ID: SparkFun-Pro-nRF52840-Mini
Bootloader: s340 6.1.1
Date: Jul 12 2019

view raw
hosted with ❤ by GitHub

Very important – update your app’s linker script:

Since your board now runs the s340 soft device, update the FLASH and RAM values in your app’s linker script:

# diff diff ~/developer/em/nRF5_SDK_15.3.0_59ac345/examples/ant/ant_plus/ant_bsc/bsc_tx/pca10040/s212/armgcc/ant_bsc_tx_gcc_nrf52.ld ~/developer/em/bia/src/ant_bsc_tx_gcc_nrf52.ld
< FLASH (rx) : ORIGIN = 0x12000, LENGTH = 0x6e000
< RAM (rwx) : ORIGIN = 0x20000b80, LENGTH = 0xf480
> FLASH (rx) : ORIGIN = 0x00031000, LENGTH = 0x000F4000-0x00031000
> RAM (rwx) : ORIGIN = 0x20002000, LENGTH = 0xf480

view raw
hosted with ❤ by GitHub

The new values are not black magic. They’re documented here: https://devzone.nordicsemi.com/nordic/short-range-guides/b/getting-started/posts/adjustment-of-ram-and-flash-memory


  • RAM and FLASH addresses: https://devzone.nordicsemi.com/nordic/short-range-guides/b/getting-started/posts/adjustment-of-ram-and-flash-memory
  • MBR and boot loader info from Nordic: https://infocenter.nordicsemi.com/index.jsp?topic=%2Fsds_s132%2FSDS%2Fs1xx%2Fmbr_bootloader%2Fmbr_bootloader.html&cp=3_4_1_0_11
  • Reading boot loader settings: https://infocenter.nordicsemi.com/index.jsp?topic=%2Fug_nrfutil%2FUG%2Fnrfutil%2Fnrfutil_settings_generate_display.html&cp=6_5_6
  • Usage of MBR params: https://devzone.nordicsemi.com/f/nordic-q-a/22329/mbr-params-page
  • Segger J-Link Mini: https://www.adafruit.com/product/3571


Random notes below, don’t follow any of it, or execute any commands from here on out. You’ve been warned.

FLASH and RAM for s340 6.1.1:

S340- 6.1.1

Min RAM start: 0x20002000

Flash start: 0x31000


Generate boot loader settings:

(nrfutil) h2:nrfutil jude$ nrfutil settings generate –family NRF52840 –softdevice ../nRF5_SDK_15.3.0_59ac345/components/softdevice/s112/hex/s112_nrf52_6.1.1_softdevice.hex –bootloader-version 1 –bl-settings-version 1 a.hex


$ git status

h2:Adafruit_nRF52_Bootloader jude$ git status
On branch master
Your branch is up to date with 'origin/master'.
Changes not staged for commit:
(use "git add <file>…" to update what will be committed)
(use "git checkout — <file>…" to discard changes in working directory)
modified: Makefile
modified: src/main.c
Untracked files:
(use "git add <file>…" to include in what will be committed)
no changes added to commit (use "git add" and/or "git commit -a")
h2:Adafruit_nRF52_Bootloader jude$

view raw
hosted with ❤ by GitHub

Installing the Nginx Ingress Controller via Helm to a K8s cluster with RBAC enabled

A lot of posts describe how to do this, but are fairly outdated, and do not mention the last supported K8s version. Here’s a tried and tested way to do so via Helm. This has been tested on GKE, with the Kubernetes master version 1.9.7-gke.6:

    1. Create the service account for Tiller – the Helm server
      $ kubectl create serviceaccount --namespace kube-system tiller
    2. Create the cluster role
      $ kubectl create clusterrolebinding tiller-cluster-rule --clusterrole=cluster-admin --serviceaccount=kibe-system:tiller
    3. Apply the RBAC role
      1. Create tiller.yml with the following content
        kind: ClusterRoleBinding
        apiVersion: rbac.authorization.k8s.io/v1beta1
          name: tiller-clusterrolebinding
        - kind: ServiceAccount
          name: tiller
          namespace: kube-system
          kind: ClusterRole
          name: cluster-admin
          apiGroup: ""
      2. Apply this
        $ kubectl create -f tiller.yaml
    4. Initialise Helm
      helm init --service-account tiller --upgrade
    5. Wait until the tiller-deploy service is running
      $ while ! kubectl get pod -n kube-system | grep tiller-deploy | grep Running &> /dev/null; do
        echo "Waiting for the tiller-deploy pod to be ready..."
        sleep 1
    6. Install the Nginx Ingress Controller
      helm install --name nginx-ingress stable/nginx-ingress --set rbac.create=true
    7. Have fun!

Inspired from Bitnami.

Read the ongoing issue here.

IntelliJ on steroids with G1 GC

Lately, I noticed that IntelliJ started to pause for quite some time during it’s GC cycles, and that it was very frequent when I was editing three files (over 1.2k LOC each) split vertically.

The current version of IntelliJ runs on a bundled version of Java 1.8, who’s default garbage collector is Parallel GC. While this works for most people, it didn’t for me.

After a ton of reading up on how GC works, and the fine tuning parameters for G1, I put it to use. Here’s a copy of my idea.vmoptions file:


view raw
hosted with ❤ by GitHub

There was an instant performance boost in the IDE – it was far more responsive than ever before. The pauses have disappeared, and it’s super snappy :)

Note: As a general rule of thumb, don’t increase the maximum memory allocated to the IDE beyond 2 gigabytes – it’s just not worth it.

How to tunnel all traffic from your iOS device to your own server via IPSec

TL;DR: A DigitalOcean droplet, strongSwan, and a custom Configuration Profile for iOS routes all the traffic from my iPhone via my droplet. Why? Just because I can.

Note: This setup does not require you to download Apple Configurator and switch your iPhone into Supervised mode (we will create a configuration profile by hand instead, and install it on the iPhone).

Configure strongSwan by following all the instructions here

  1. Ignore the part about configuring the firewall, we’ll do this later
  2. Ensure strongswan starts on boot via chkconfig
    chkconfig --add strongswan
    chkconfig strongswan on
    # Verify
    chkconfig --list strongswan
  3. You don’t need to install any certificates on your iPhone/iPad/Mac as we’re using a pre-shared key (PSK) instead of a certificate based client authentication mechanism

Allow traffic to be forwarded from your server by adding the two iptables rules here

Be sure to modify the network in the two iptables commands (it should match the one specified in your strongSwan config)

Save the two rules which you’ve just added

service iptables save

Open up UDP ports 500 and 4500 for your instance if required (AWS/DigitalOcean/etc)

Adapt the following Configuration Profile for your iOS device

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<string>VPN Configuration</string>

view raw
hosted with ❤ by GitHub

Replace the following variables with reasonable values for your setup:

MY_PROFILE_NAME                  - Only used for display purposes
MY_DOMAIN                        - Just for scoping
MY_STRONGSWAN_SERVER_IP_ADDRESS  - Your server's IPv4 address
MY_ACCOUNT_NAME                  - See /etc/strongswan/ipsec.secrets
MY_ACCOUNT_PASSWORD              - See /etc/strongswan/ipsec.secrets
MY_INITIALS                      - Your initials (eg: JP)

Once you’ve updated the content of the XML file above, rename the file to VPNConfig.mobileconfig. Then, either AirDrop it to your iPhone/iPad, or transfer it by some other means.

Since we’re using a PSK, as soon as you install the profile, it’ll prompt you for the PSK. This can again be found in /etc/strongswan/ipsec.secrets.

All done! :)

Cheers on your newly established, always on VPN tunnel between your iOS device and your server!

The Configuration Profile was inspired from Thomas’s blog post here.

The Bombay Santa – An Exclusive Secret Santa Gathering

This year, for the very first time in Mumbai, LetsTuneup is hosting an exclusive, invite only Secret Santa gathering.

I got invited!

I was one of the firsts to be invited (yes!). We’re planning to have a fun filled, gift exchanging afternoon sometime during the Christmas week. The venue is yet to be declared. If you’re looking for an invite, tweet to LetsTuneup!

More information about this event is available here.

We’re looking forward to meeting all of you out there!

LetsTuneup was on Ishq 104.8 FM today!

Today marked a significant milestone in LetsTuneup’s life – we were interviewed live by RJ Sangeeta on Ishq 104.8 FM in Mumbai!

How it all came about

Yesterday, the morning show producer of Ishq FM messaged me, and asked if the RJ could talk to me the very next morning for a brief chat about LetsTuneup. The producer had read about LetsTuneup in an app review which was published by one of our fans a couple of weeks ago.

RJ Sangy hosts a morning show called Ishq Hangouts. We did a brief interview which lasted for about 12 minutes this morning.

For those of you who missed it earlier today, here’s the audio recording of the chat, split over four parts.

Personally, I’ve never been on air before. I loved my first experience, albeit I was extremely nervous just before it – the quick fix, I was listening to Mirrors by Niall Horan on a loop for about ten minutes just before the interview :)

Contributing to Go in 54 days

With absolutely zero knowledge of Go 54 days ago, I decided to contribute to the Go project. Why? Put simply, I was bored. The thrill of learning something new, and contributing to a massive OSS project like Go caught my attention.


  1. Find an issue that’s tagged as HelpWanted.
    1. There’s a “HelpWanted” tag, which is applied to issues where the Go community is looking for somebody on the outside to fix. I found one such a issue, #21216 with the topic being x/build/cmd/cl: build broken. This seemed a great place to start.
  2. Go through their Contribution Guide.
  3. Although I skipped this part at first, the commenting guide.
    1. I split the issue at hand into two parts, one that provided the resource, and the other to actually fix the reported issue.
    2. On my very first CL (change list), my commenting style varied greatly. I was asked to review the commenting guide. Read it. Seriously, read it.
  4. A must read before starting, Effective Go.
  5. Take a tour of it, in A Tour of Go.
  6. Use Gogland (I love JetBrains for their outstanding IDEs).

Learning Go from scratch was a fairly simple task. It’s just a new syntax, nothing more. Moreover, there’s always Stack Overflow to help you out. Think of SO as a passive mentor, who gives you advice when it’s asked.

I’ve got to thank a couple of people who helped me along the path, @kevinburke, @bradfitz and @andybons. They reviewed my code, and gave my changes a +2, and submitted them.

What does it feel like?

It feels like the first time you try to dive into a swimming pool. You don’t know whether you can do it, but you do it nevertheless. Getting my first two CLs accepted was a little challenging, but definitely enthralling. Talking to other like-minded people across the globe, committed to fixing issues and innovating, is a completely new experience to me. I’m now set on a path to contribute to Go, as it’s a fun weekend exercise, and moreover, just because I can.

LetsTuneup: A music chart with Arjit Singh in the lead

LetsTuneup has grown tremendously, and with it, we’ve introduced new features too. We identified that a few of our users couldn’t use the app to it’s full extent because they didn’t have music on their devices.

We’ve solved that. Users can now pick their favourite artists, powered by a location aware scoring algorithm, which recommends popular artists in their area.

Leading the recommendation list in Mumbai is Arjit Singh, followed by Eminem, Linkin Park, Coldplay and Pink Floyd. Honey Singh is #11 on the chart, and some nostalgic users love Akon, making him #28.

Arjit Singh in the lead, with Eminem, Linking Park, Coldplay and Pink Floyd following close
Arjit Singh in the lead, with Eminem, Linkin Park, Coldplay and Pink Floyd following close

Stay tuned and look forward to our next big feature, very soon.

US toy giant threatens to sue Matchbox

TL;DR: A giant American toy company threatens to sue Matchbox. Having no resources at hand, Matchbox is forced to change it’s name to Tuneup.

On the 14th of March, I received an email from Apple (via the giant toy company) with the following content:

The developer of the reported application is using the registered
Matchbox trademark in the description of the application without
authorization from the right holder. Also he is offering services by
unlawfully using the registered trademark which is causing bad
influence for the Matchbox brand values. Furthermore, this application is
not authorized from the right holder. This is a trademark infringement
causing damages to our client xxxxxxxx, the right holder for the
Matchbox intellectual property. I would kindly ask you to proceed and
remove the infringement/application from all stores worldwide.

This company, had the trademark for Matchbox registered under the following classification codes: US 001 002 003 005 021 022 023 026 029 036 037 038 039 041. Each and every G & S description was related to toy products, fabrication, stationery products, and manufacturing processes.

Looking up these classification codes on the United States Patent and Trademark Office’s website, absolutely none of them are for computer software, or anything that even crosses paths with Matchbox.


The million dollar question: If there was absolutely no case for “confusingly similar”, why did Matchbox change it’s name?

It’s simple really. Big guys always bully the small ones. They threatened to sue us if we didn’t stop using it. Even after proving to them that it wasn’t even remotely infringing to their use of Matchbox, they wouldn’t budge. Had I fought them legally, I would’ve won. Easy peasy. However, due to the lack of resources, I had no choice but to change it’s name.

If I had to call Matchbox anything other than Matchbox, I’d call it Tuneup. “Tuneup” is imagined by Joelle Fernandes, the co-founder of Let’s Tuneup.