14.04.2023 aktualisiert
DS
100 % verfügbar
Azure Data Engineer
Frankfurt am Main, Deutschland
Weltweit
M.Sc. in InformatikSkills
AzureAzure Data FactoryAzure DatabricksAzure SynapseMicrosoft Azure Logic AppDatenbankSQLPythonETL DWHData IntegrationData MigrationsDelta lakeAzure StorageAzure Sql
- Microsoft Azure
- Azure Data Factory
- Azure Synapse
- Azure Databricks
- Azure Logic Apps
- Azure Functions
- Azure DevOps
- SQL
- data migration
- data integration
- cloud
- ETL
- DWH
- Python
- Jira
- Confluence
- Scrum
Sprachen
BulgarischMutterspracheDeutschverhandlungssicherEnglischverhandlungssicher
Projekthistorie
- Design and development of scalable data ingestion pipelines to transform wide variety of Big Data datasets using Azure Synapse in the Enterprise Data Lake
- Data analysis and transformation using Spark, Pyspark and Notebooks in Azure Synapse
- Working with Big Data in parquet and delta formats
- Design tables using dedicated SQL pool in Azure Synapse Analytics
- CI/CD in Azure DevOps
- Research, analyze and implement technical approaches
Verantwortlich für die Datenmigration und Datenintegration zwischen zwei Systemen (On-Premise in die Microsoft Azure Cloud)
Technologien:
Technologien:
- Azure Data Factory
- Azure Functions
- Azure DevOps
- Azure Logic Apps
- Microsoft SQL
- Python
- Rest API Git
- Soap, XML
- JSON
- Jira
- Confluence
- Scrum
- Design and implement ETL processes using Azure Databricks to move data from various sources to a data warehouse
- Develop and maintain data pipelines to ensure reliable and efficient Scheduling orchestration using Azure Data Factory
- Implementation of PySpark/SQL code to transform and process data, and to implement complex business logic using Databricks Notebooks
- Troubleshoot and resolve issues related to data processing, data quality, and system performance
- Develop and maintain technical documentation for data processing pipelines and data solutions
- Troubleshoot and resolve data-related issues in a timely manner