Azure Data Engineer
Design, develop, and maintain scalable ETL pipelines using Azure Services to process, transform, and load large datasets into AWS Datalake or other data stores.
Collaborate with cross-functional teams, including data architects, analysts, and business stakeholders, to gather data requirements and deliver efficient data solutions.
Design, implement, and maintain data pipelines for data ingestion, processing, and transformation in Azure.
Work together with data scientists and analysts to understand the needs for data and create effective data workflows.
Create and maintain data storage solutions including Azure SQL Database, Azure Data Lake, and Azure Blob Storage.
Utilizing Azure Data Factory or comparable technologies, create and maintain ETL (Extract, Transform, Load) operations.
Implement data validation and cleansing procedures to ensure the quality, integrity, and dependability of the data.
Improve the scalability, efficiency, and cost-effectiveness of data pipelines.
Monitor and resolve data pipeline problems to guarantee consistency and availability of the data.
Key Skill Sets Required:
Experience in designing and hands-on development in cloud-based analytics solutions. Expert-level understanding of Azure Data Factory, Azure Synapse, Azure SQL, Azure Data Lake, and Azure App Service is required. Experience in designing and building data pipelines using API ingestion and streaming ingestion methods. Knowledge of DevOps processes (including CI/CD) and Infrastructure as Code is desirable. Strong experience in common data warehouse modeling principles including Kimball and Inmon. Knowledge of Azure Databricks, Azure IoT, Azure HDInsight + Spark, Azure Stream Analytics, and Power BI is desirable. Working knowledge of Python is desirable. Experience developing security models.
#J-18808-Ljbffr
Experience in designing and hands-on development in cloud-based analytics solutions. Expert-level understanding of Azure Data Factory, Azure Synapse, Azure SQL, Azure Data Lake, and Azure App Service is required. Experience in designing and building data pipelines using API ingestion and streaming ingestion methods. Knowledge of DevOps processes (including CI/CD) and Infrastructure as Code is desirable. Strong experience in common data warehouse modeling principles including Kimball and Inmon. Knowledge of Azure Databricks, Azure IoT, Azure HDInsight + Spark, Azure Stream Analytics, and Power BI is desirable. Working knowledge of Python is desirable. Experience developing security models.
#J-18808-Ljbffr
