My personal project for data engineering zoomcamp
This utilizes a static IP address in GCP to pull data from the Clash of Clans API, store it and serve it via data studio. There is also a API client with a sleep of functions you can use to pull, store, and view CoC facts and dimensions.
Easy-to-use Python library for managing files and data in different environments
Scripts used for analyzing GitHub commit locations to create a map visualization
A platform for users to upload and share the responses from law enforcement agencies to their traffic violation reports in Taiwan. This system aims to increase transparency and public oversight of traffic law enforcement.
Explore efficient big data matching techniques
Data ingestion from Google BigQuery to PostgreSQL
Market Data Ingestion and BigQuery Integration project
A repo containing auto-triggered Airflow ETL activities for datasets located on GCP storage that flattens and creates analytical views on Big Query
Webscraping dos dados do site Mercado Livre. Esses dados são exportados para uma base de dados BigQuery e algumas informações relevantes são obtidas através de consultas SQL.
Scrape Premier League Data and load it into Google BigQuery
As part of my Data Engineering professional development, I am developing an array of data pipelines which extract, transform and load the data from various sources such as this Quandal API, CSV, JSON and database such as Google BigQuery, Microsft SQL Server and PostgreSQL, just to name a few.
A word cloud based on browser mentions in GitHub commit messages.
Welcome to the Spotify Insights Data Pipeline Project where I analyze data from my Spotify listening history ~
This is the second version of the Google Serch Trends API. Having completed the initial testing of the first version. The next step was to create a series of functions in line with the principles of object-oriented programming. Initially was created in a linear format as a proof of concept. Later split the code into five main functions.
Add a description, image, and links to the big-query topic page so that developers can more easily learn about it.
To associate your repository with the big-query topic, visit your repo's landing page and select "manage topics."