AWS/Azure Solutions Data & Data Mesh Architect / Senior Platform Engineer

• Develop and manage AWS & Azure-based infrastructure solutions,
ensuring optimal performance and security.
• Design and launch bespoke data/analytics platforms and a Unified Data
Ecosystem tailored for the European (DE), U.S, and Chinese markets,
enhancing data processing capabilities.
• Develop and implement robust DevSecOps, DataOps, and MLOps
practices, automating workflows, optimizing CI/CD pipelines, and enabling
efficient, scalable, and secure operations across data.
• Develop a comprehensive lakehouse architecture, incorporating data
layers, scalable streaming data pipelines, and efficient data modeling
frameworks to enhance data structuring and accessibility
• Architect and implement Data Mesh architecture to enhance data
governance, interoperability, and domain-driven data ownership across the
organization
• Formulate and execute a robust security and disaster recovery strategy,
strengthening system resilience.
• Enhance data and analytic infrastructure, focusing on scalability, efficiency,
and continuous improvement.
• Design cloud-centric solutions, emphasizing PaaS, SaaS, serverless
technologies, and microservices to optimize service delivery.
• Collaborate with stakeholders to align solution designs, ensuring seamless
architecture, development, and operational processes.
• Oversee the development and maintenance of secure, scalable
microservices, monitoring system integrity and performance.
• Design and Implement scalable data platforms with secure
communication using Service Mesh technologies, ensuring efficient and
reliable data processing across distributed systems..
• Spearhead the development of a comprehensive Data Governance
Framework, establishing robust standards for data quality, metadata
management, and regulatory compliance to ensure consistent practices and
secure, enterprise-wide data access.
• Strategize and conduct extensive data migrations, securely transitioning
terabytes of data.
Technology

Technologies

• Technologies
  • AWS: Landing Zone, Load Balancer, Control Tower, Direct Connect, Guardrails,, GuardDuty, Config, Redshift (Spectrum, Multi-Cluster & Data Sharing), S3, SQS, Lambda, Glue (ETL jobs, Catalog, Crawlers), Athena, EKS, DynbamoDB, SageMaker, BedrocK, Lex, Polly, Textract, Data Pipeline, DynamoDB, API Gateway, SecurityHub, CloudTrail, Kinesis, Transit Gateway, Site-to-Site VPN, VPCs, PostgreSQL, VPCs, Step Functions, QuickSight, OpenSearch.
  • Azure: Azure AD, DevOps Pipelines, Data Factory, Data Explorer, Data Lake Storage, Synapse Analytics, Load Balancer, Cosmos DB, Functions, AKS, Event Hub, Service Bus, App Services, vNet (VPN Gateway, Virtual WAN, Private Link), Key Vault, Storage Accounts, Security Center, Defender for Cloud, Sentinel, WAF, Firewall, DDoS Protection, Managed HSM, Policy, ExpressRoute, Monitor, Log Analytics, Application Insights.
  • Big Data & Streaming: Kafka (Confluent, Kafka Streams, RabbitMQ, Kafka Connect), Apache Flink, Iceberg, Arrow, Avro,, Parquet.
  • Data Processing, Visualization & Governance: Databricks (Delta Lake, Unity Catalog, Lakehouse Federation, Auto Loader, Data Sharing, Databricks Jobs, Model Monitoring, Feature Store, RAG, LLM Integration), Mlflow, dbt-core, Hadoop, Airflow, Dremio, Trino, Starburst, Colibra.
  • DevOps & Infrastructure: Terraform, Bicep, Helm Charts, Istio.
  • Containerization & Orchestration: Kubernetes, OpenShift, Docker.
  • Monitoring & Logging: OpenTelemetry, Dynatrace.
  • Graph & Time Series Databases: Amazon Neptune, Azure Cosmos DB (con API for Gremlin), InfluxDB
  • CI/CD & Version Control: GitHub Actions, ArgoCD, Argo Workflows
  • Collaboration Tools: Jira, Confluence.
  • Languages and Frameworks: Python (FastAPI, scikit-learn), PySpark, PyFlink, Java.
• •Frameworks & Agents: LangChain, LangGraph, LangSmith, CrewAI, AutoGen, Agent Workflows, LangChain Expression Language
  •Vector Databases: Pinecone, pgvector, FAISS, OpenSearch, Weaviate
  •Cloud & AI Services:
  • AWS: Bedrock, Lex (Conversational AI), Comprehend (NLP), Polly (Speech), Rekognition (Vision), Lambda, Step Functions, DynamoDB, API Gateway, S3
  • Azure: Azure OpenAI, Cognitive Search, Cognitive Services (Language, Vision, Speech), Azure ML, Functions, Cosmos DB, Event Hub, Key Vault
  •Cross-cloud: OpenAI API (ChatGPT, Embeddings), Anthropic Claude.
  •Observability & CI/CD: OpenTelemetry, LangSmith (Tracing & Evaluation), ArgoCD, MLflow, GitHub Actions, Terraform
  •Languages & Tools: Python (FastAPI, Prompt Engineering), SQL, Pandas, Docker, Kubernetes