Log In
Sign Up
Job Description:
- Professional experience in Data Engineering using Google Big Query (GBQ) and Google Cloud Platform (GCP) data sets and building data pipelines.
- Hands on and deep experience working with Google Data Products (e.g. BigQuery, Dataflow, Dataproc, Dataprep, Cloud Composer, Airflow, DAG etc.).
- Python Programming Expert PySpark, Pandas.
- Experience in Airflow (Create DAG, Configure the variables in Airflow, Scheduling).
- Big Data technologies and solutions (Spark, Hadoop, Hive, MapReduce) and multiple scripting and languages (YAML, Python).
- Experience in DBT to create the lineage in GCP. – Optional
- Worked in Dev-Sec-Ops (CICD) environment.
- Design and develop the ETL ELT framework using BigQuery, Expertise in Big Query concepts like Nested Queries, Clustering, Partitioning, etc.
- Experience in Data Integration, Data Transformation, Data Quality and Data Lineage tools.
- Should be able to automate the data load from Big Query using APIs or scripting language.
- E2E Data Engineering and Lifecycle (including non-functional requirements and operations) management.
- E2E Solution Design skills – Prototyping, Usability testing and data visualization literacy.
- Experience with SQL and NoSQL modern data stores.
Manage Consent
Functional Always active
The technical storage or access is strictly necessary for the legitimate purpose of enabling the use of a specific service explicitly requested by the subscriber or user, or for the sole purpose of carrying out the transmission of a communication over an electronic communications network.
Preferences
The technical storage or access is necessary for the legitimate purpose of storing preferences that are not requested by the subscriber or user.
Statistics
The technical storage or access that is used exclusively for statistical purposes.
The technical storage or access that is used exclusively for anonymous statistical purposes. Without a subpoena, voluntary compliance on the part of your Internet Service Provider, or additional records from a third party, information stored or retrieved for this purpose alone cannot usually be used to identify you.
Marketing
The technical storage or access is required to create user profiles to send advertising, or to track the user on a website or across several websites for similar marketing purposes.