We wanted to be able to dynamically notify Data Magicians when a task had completed, without them having to refresh their browser screen constantly. Implementing websockets allowed us to achieve this.

We no longer need to manually track the state of a table, when it was created, when it was updated, which data pipeline last touched it …. all these data points are available by doing a simple call to the logging and bigquery api. Under the covers the google cloud platform is already tracking everything we need … every insert, update, delete, create, load, drop, alter is being captured

In the word of agile, there are three common testing techniques that can be used to improve our testing practices and to assist with enabling automated testing.

This is the first of a series of articles detailing how we built a platform to make data fun and remove complexity for our users