Data Engineering (Azure Databricks)

We are seeking an experienced Azure Databricks Developer to join our data engineering team in Pittsburgh, PA. The ideal candidate will have strong expertise in building scalable data pipelines and transforming large datasets using Azure Databricks, Spark, and other Azure data services. You will work closely with data architects, analysts, and business stakeholders to design and implement high-performance data solutions that drive business insights.

Responsibilities:

• Design, develop, and optimize data pipelines and ETL/ELT workflows using Azure Databricks, Spark, and Azure Data Factory.
• Integrate data from multiple sources including on-premise and cloud systems (e.g., Azure Blob Storage, Azure SQL, Synapse).
• Implement and maintain scalable data lakehouse architectures leveraging Delta Lake.
• Collaborate with data scientists and BI teams to prepare clean and usable datasets for analysis and machine learning models.
• Monitor and optimize pipeline performance, ensuring reliability, data quality, and scalability.
• Write efficient PySpark or Scala code within the Databricks environment.
• Ensure compliance with data governance, security, and privacy policies.