agiledata Blogs

Because sharing is caring

Can we use an Information Product Canvas image to start the data design process?

Can completed Information Product Canvas image help with the initial design of the data environment to deliver that Information Product?

Overall the Disco outputs were not good enough to be used as a data design (not sure it ever will be perfect), so some more iteration to be done.

But the output it did generate was an encouraging start.

ELT without persisted watermarks ? not a problem
ELT without persisted watermarks ? not a problem

We no longer need to manually track the state of a table, when it was created, when it was updated, which data pipeline last touched it …. all these data points are available by doing a simple call to the logging and bigquery api. Under the covers the google cloud platform is already tracking everything we need … every insert, update, delete, create, load, drop, alter is being captured