16.07.2024 aktualisiert
100 % verfügbar
Data Architect Stream Processing/Big Data, Data Engineer, Software Engineer
Köln, Deutschland
Deutschland
Skills
Apache SparkawsScalaData EngineeringSolution Architecture Data ArchitectureBig DataApache KafkaStream Processing
Stream Processing, DevOps, Apache Kafka, Kafka Streams, Ansible, Azure Cloud, IoT, Apache Spark, Apache Cassandra, Scala, cloud, data storage, hadoop, Apache, Cassandra, Big Data, kubernetes, SOA, Analytics, AWS, ETL, Alteryx, Spark, HDF, Hive, Amazon Web Services, Software as a Service, AWX, Terraform, Docker, OpenStack, Streams, Spring Framework, bug fixes, Cloud implementations, Apache Zeppelin, Hashicorp, Brocolli, persistent storage, Cloudera, Kerberos, API, Code Review, Spring Boot, HDFS, Prometheus, Grafana, Message Broker, Event Sourcing, CQRS, functional programming, stateful, Flume, Impala, sensor, Data Warehouse, Java, PostgreSQL, Jenkins, Google Cloud, CI/CD, Microservices, Node.js, AWS EMR, R Studio, S3, Apache Hadoop, Apache Parquet, JSON, JSON schema, Streaming, Parquet, data integration, development environment, VPC, EC2, AutoScaling, EMR, Akka Streams, SSTable streaming, Cassandra Cluster, Webshop, Google Analytics, SQL, data ingestion, Apache Kudu, Apache Impala, Apache Hive, Kudu, virtualization, MapR, Logstash, Elasticsearch, cluster layout, automated deployment, REST API, InfluxDB, Machine Learning, ReactJS, Time series analysis, React, JS, Apache Flume, Apache Avro, Apache HBase, HBase, Dependency Injection, Google, Guice, SaaS, Apache YARN, Hashicorp Nomad, YARN, Service Discovery, AngularJS, SBT, Angular, Avro, HTTP API, Reverse Engineering, libs, MySQL, Spring MVC, REST application, REST APIs, frontend, Puppet, Ubuntu 14.04, OS, Load Balancing setup, Play Framework, Hibernate, Spring MVC Application, PHP, Wordpress, backend, WebSocket, Doobie, Shapeless, TypeScript, GraphQL, OAuth2, CentOS, Github, web application, JDBC, data handling, Stream Analytics, Kibana, WebSockets, Spotify, ES6, BEM, SASS, Docker Compose, database, Lucene
Sprachen
DeutschMutterspracheEnglischgut
Projekthistorie
Customer Industry Retail, E-Commerce
Project Consultant Stream Processing Architecture, Big Data Introduction for User Tracking Implementation
and BI
Roles Consultant, Architect, DevOps Engineer, Data/Software Engineer
Focus Workshop, Architecture, Implementation
Team Languages German/English
Toolset Hortonworks HDP/HDF, Apache Spark, Apache Hive, Apache Kafka, Apache Cassandra, Ansible, Amazon Web
Services, Scala, Akka Streams, Kafka Streams
Description of project
* Workshops about Hadoop and its use cases
* Concept and Implementation of Big Data processing platform
* Automated deployment of Hortonworks HDP/HDF
* ETL and aggregation pipeline design
* Implementation of pipelines
* Setup Spark on Kubernetes with Hive Metastore
Project Consultant Stream Processing Architecture, Big Data Introduction for User Tracking Implementation
and BI
Roles Consultant, Architect, DevOps Engineer, Data/Software Engineer
Focus Workshop, Architecture, Implementation
Team Languages German/English
Toolset Hortonworks HDP/HDF, Apache Spark, Apache Hive, Apache Kafka, Apache Cassandra, Ansible, Amazon Web
Services, Scala, Akka Streams, Kafka Streams
Description of project
* Workshops about Hadoop and its use cases
* Concept and Implementation of Big Data processing platform
* Automated deployment of Hortonworks HDP/HDF
* ETL and aggregation pipeline design
* Implementation of pipelines
* Setup Spark on Kubernetes with Hive Metastore
Customer Itellity services Ltd
Project Crawler Reimplementation
Roles Architect, Software Engineer
Focus Concept, Implementation
Team Languages German
Toolset Akka Streams, Akka HTTP
Description of project
Reimplementation of existing application due to external API changes.
Implementation has been done using Akka Streams to overcome problems and complexity of existing,
mostly side effecting, imperative style implementation.
Project Crawler Reimplementation
Roles Architect, Software Engineer
Focus Concept, Implementation
Team Languages German
Toolset Akka Streams, Akka HTTP
Description of project
Reimplementation of existing application due to external API changes.
Implementation has been done using Akka Streams to overcome problems and complexity of existing,
mostly side effecting, imperative style implementation.
Customer Vaillant GmbH
Project Realtime Streaming Architectures for IoT with Kafka - Consultancy
Roles Consultant, Architect, Data Engineer
Focus Workshop, Architecture and Code Review
Team Languages German
Toolset Apache Kafka, Spring Boot, Hadoop HDFS, Apache Spark, Akka, Akka Stream, Kafka Streams, Docker,
Kubernetes, Prometheus, Grafana
Description of project
* Usage as Message Broker as basis for realtime streaming architectures
* Kafka Internals
* Deployment automation
* Kubernetes
* Plain VM via Ansible and Terraform
* Operations
* Patterns/Solutions, Event Sourcing/CQRS
* Hadoop Data Lake integration
* Intro into functional programming
* High available, distributed, stateful Stream Processing
Project Realtime Streaming Architectures for IoT with Kafka - Consultancy
Roles Consultant, Architect, Data Engineer
Focus Workshop, Architecture and Code Review
Team Languages German
Toolset Apache Kafka, Spring Boot, Hadoop HDFS, Apache Spark, Akka, Akka Stream, Kafka Streams, Docker,
Kubernetes, Prometheus, Grafana
Description of project
* Usage as Message Broker as basis for realtime streaming architectures
* Kafka Internals
* Deployment automation
* Kubernetes
* Plain VM via Ansible and Terraform
* Operations
* Patterns/Solutions, Event Sourcing/CQRS
* Hadoop Data Lake integration
* Intro into functional programming
* High available, distributed, stateful Stream Processing