the output after parsing as it would look from a Jupyter notebook

An important task in Data Engineering is writing efficient scalable pipelines that move data from A to B. There are many GUI-tools out there and they are great because there is a real shortage of Data Engineers at the moment.

Being Data Engineers however we should not be limited by what any tool can do for us, locked in by any vendor or having to spent time figuring out how a brandnew hip GUI works every few months.

Therefore it is preferable to build a toolbox with migration-scripts for the most common use-cases and this series of articles is intended…


Firebase NoSQL database available via Google Cloud Platform

Migration a MySQL Database to GCP Firebase using Python and Pandas

I have a little VM on a well-known Developer Cloud which I love for quick prototyping for new and prospective clients.

It has all my Data Science and Machine Learning Libraries installed, a Flask Webserver, built on uWSGI behind an Nginx Reverse Proxy, for storage I have installed and connected MongoDB, Redis and MySQL.

Therefore, it is easy for me to quickly scrape some data or train an ML model and put it in a somewhat presentable form onto this server to demonstrate what I have built.

This works…

Sanardi Azam

Google Cloud Certifed Data Engineer Professional

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store