Data Engineer - Remote (US) - Full Time Only

Role: Data Engineer

Location: Remote (Candidate should be comfortable following PST Time Zone)

Job Type: Full Time

Active LinkedIn ID is a must and must match with the resume.

Must Have Skills

• Azure Databricks, Apache Spark, Snowflake, Data Governance, Unity Catalog, dBT, Microsoft Purview, Kafka, SQL, Python

Experience And Qualifications

• Bachelor’s or Master’s degree in Computer Science, Engineering, or a related field.
• 10+ years of experience in data engineering / data architecture, with deep hands‑on expertise in Microsoft Azure.
• Expert‑level experience with Azure Databricks and Apache Spark (PySpark, Spark SQL).
• Delta Lake, Unity Catalog, Job orchestration and performance tuning.
• Strong experience with Snowflake on Azure, including:
• Schema and data model design
• Performance and cost optimization
• RBAC, Streams, and Tasks
• DBT (Core or Cloud) expertise:
• Project structure and best practices
• Tests, exposures, macros
• Strong fundamentals in SQL and Python.
• Proven experience building ETL/ELT pipelines using Azure Data Factory (ADF) and Azure Synapse / Microsoft Fabric pipelines.
• Familiarity with streaming platforms such as Event Hubs and/or Kafka.
• Solid understanding of Lakehouse and data warehousing architectures, dimensional modeling, medallion architecture patterns, and data quality frameworks.
• Strong knowledge of security and governance, including Microsoft Purview (catalog, lineage), data masking and PII handling, managed identities, and private networking.
• Excellent communication and documentation skills, with the ability to create architecture and design documents and present to both technical and business stakeholders.
• Consulting / agency experience, with the ability to lead multiple concurrent projects.
• Willingness to travel as required.

Nice To Have

• Experience with Azure ML, feature stores, or serving ML pipelines from Databricks or Snowflake.
• Infrastructure as Code (IaC) using Terraform or Bicep.
• Containerization experience with Docker.
• Observability and data quality tooling, such as Great Expectations / Delta Expectations, Databricks Quality Flows, Monte Carlo, Datadog.

Preferred Certifications

• Databricks Certified Data Engineer Professional or Databricks Architect.
• Microsoft Azure: DP-203 – Azure Data Engineer, Azure Solutions Architect Expert, Azure Security Engineer Associate.
• Snowflake: SnowPro Core, SnowPro Advanced.

Job Description / Responsibilities

• Partner with client stakeholders to translate business objectives into scalable Azure data architectures and delivery roadmaps.
• Architect and implement lakehouse and data warehouse solutions using Azure Databricks (Delta Lake, Unity Catalog), Databricks Bundle, Snowflake on Azure, dbt (Core / Cloud), ADLS Gen2, and medallion architecture patterns.
• Design and build robust ingestion and transformation pipelines using Azure Data Factory (ADF) and Microsoft Fabric / Synapse pipelines.
• Orchestrate ELT workflows using Databricks Jobs, Delta Live Tables, and dbt models.
• Establish best practices for performance and cost optimization, including cluster sizing and autoscaling, Photon and SQL Warehouses, Snowflake virtual warehouses, caching, file layout optimization, and Z‑ordering.
• Drive observability and monitoring using Azure Monitor, Log Analytics, and Databricks metrics.
• Implement robust data governance and security using Azure Authentication, Unity Catalog, RBAC / ABAC, managed identities, private endpoints and VNet injection, and Azure Key Vault–backed secrets.
• Lead code and design reviews, setting standards for PySpark, SQL, and dbt, unit and integration testing, and data quality checks (expectations and constraints).
• Implement CI/CD pipelines using Azure DevOps or GitHub Actions.
• Guide multi‑domain programs (Healthcare, Retail, BFSI, or similar).
• Mentor data engineers and ensure high‑quality, production‑ready deliverables.
• Evangelize solutions through clear documentation, reference architectures, and executive and stakeholder presentations.

#J-18808-Ljbffr