Can We Bug You for a Few Minutes?

A few hundred of you have been able test our very first private beta of the ecosystem back in March. Since then we’ve been building towards the launch of our public beta for both iOs and Web app and we can’t wait to share for you to start collecting your medical data, get predictive insights and join data trials! So stay tuned because it won’t be that long before you receive that email telling you download the app and Get Started.

Meanwhile we have News from the Trenches with the case of a mysterious, deadly and totally fun bug!

In addition to our iOS app ready to ship early this Summer, we are building a brand spanking new web portal to enable our users to use the web or their Android phone to train their AI! This web portal is a Next.js based, Redux + Router isomorphic app that allows us to rapidly develop. In the meantime our infrastructure team has been with load testing going (especially with our importer framework using dummy importers ), designing our kubernetes cluster and several other critical tasks. Next.js brings many things to the table, with trade offs on rendering clarity. The lack of clarity on how components and services render (Server Side Rendering, using window, etc) and are built is augmented by the stark difference of the application in development v.s. production.

Often with routing bugs, we have found it to be to due to our misunderstanding of the next/Router and redux integration. As we have been using Next.js we have been getting better and better at preventing state mutations and other issues caused by eventing. The team has grown in skills phenomenally with the recent work, especially considering we have added 4 new team members in rapid succession.

That is why it was a bit of surprise when we were hit by a very strange issue on our development web application. When the frontend team deployed the web app for a recent internal review we saw cancelled XHR requests for styles, assets, images and bundles.

It left our app looking like this!

The team immediately felt that it was a major routing issue and after a few days of investigation we had no clue what exactly was causing the issue. We kept scouring the docs, trying various implementation of the router, etc.

Eventually, we all noticed a weird request that we had passed off as a failed or weird webpack call.

A very weird Request

The request happened to be big. Very big. It looked something like this:

https://{app_url}/iVBORw0KGgoAA... many characters later … 1LJjY3KDewOmGiTA00=

If you notice at the end of the URL there is an ‘=’ character. Purely out of curiosity I decided to copy the request as curl and see if it was actually base64 encoded. I was expecting that it was a webpack javascript bundle that we had all assumed earlier.

Running the follow command:

$ cat ~/Desktop/whatisthis.txt | base64 — decode

I was surprised to see:

And from years of dealing with base64URL images I realized this could be an image! So loaded the base64 decoded data into a file and guessed at the image prefix.

$ cat ~/Desktop/whatisthis.txt | base64 — decode > maybeImage.png

Sure enough, it was …… a wonderfully large high quality … testing image!

Our infrastructure team has been doing load testing on our importer frameworks and decided to make a “fake importer” to test out our platform. And sure enough they forgot to include the prefix need for an HTML img tag to actually render the base64! (The prefix being “data:image/png;base64,”)

What was happening was the img tag was making an HTTP request with the base64url in the src property. This was a large URL request that was failing and causing other ajax requests to cancel.

Needless to say this bug revealed to us a critical hole in our framework which we will be using to build more reliable and solid products. Additionally, the amount of fun we had following this rabbit hole was phenomenal!