05.10.2023 aktualisiert
100 % verfügbar
Data Engineer - Cloud | ETL | Event Streaming | Kafka | DevOps
Stuttgart, Deutschland
Weltweit
MSc Computer ScienceSkills
I am a Data Engineer with 5 years of practical experience developing scalable ETL pipelines and data processing applications, both batch and real-time streaming.
I enjoy developing software and architectures with best practices in mind, striving for reliable and maintainable solutions.
I love to work in agile environments, solving technological challenges and delivering business value as early as possible at the same time.
My focus:
I enjoy developing software and architectures with best practices in mind, striving for reliable and maintainable solutions.
I love to work in agile environments, solving technological challenges and delivering business value as early as possible at the same time.
My focus:
- Python, Java, Scala
- Spark (Batch / DStreams / Structured Streaming)
- Kafka (Producer-Consumer API / Kafka Streams / Kafka Connect)
- Cloud technologies (terraform, AWS)
- Container technologies (docker / kubernetes)
- Tooling for every part of the software lifecycle (git, CI/CD, extensive integration testing, test automation, logging, monitoring)
Sprachen
DeutschMutterspracheEnglischverhandlungssicherFranzösischgut
Projekthistorie
Data from various source systems is ingested from Kafka, then enriched and transformed for the requirements of target systems in Kafka Streams Java microservices. Kafka Connect is used to transfer the data to the target systems, e.g. via JDBC.
- Apache Kafka (Kafka Streams, Kafka Connect)
- Java, Quarkus
- Red Hat OpenShift
- Splunk, SignalFX
Customer data (e.g. sales data, web site click stream) was ingested as a data stream from various source systems and integrated in real-time into a marketing automation platform. User characteristics such as customers content preferences were extracted in real-time, enabling the customer relations team to achieve precise customer segmentation and provide customers with highly personalized communication.
- Apache Kafka (Kafka Streams)
- Migration of legacy AWS Kinesis data
- Apache Spark (DStreams API, Structured Streaming)
- Kubernetes, kustomize
- CI/CD (Gitlab CI)
Customer data from various source systems was integrated with target systems of company-internal stakeholders in the commercial department to enable reporting and analysis.
- Created reliable data pipeline applications to ingest the data in a data lake, process and combine the data and load it into database systems
- Comprehensive concept for complying with GDPR and company privacy policies for public clouds using encryption
- Kubernetes / spark-on-k8s, kustomize
- Airflow for workload scheduling
- AWS cloud technologies (S3, RDS), terraform
- Pyspark
- CI/CD (Gitlab CI)
Zertifikate
Certified Developer for Apache Kafka
Confluent2020