Cloud Data Engineer

Kforce has a client in Boston, MA that is seeking a Cloud Data Engineer. Lakehouse Architecture (Apache Iceberg):

• Design and build Iceberg-based data lakes with ACID-compliant, versioned datasets
• Implement Iceberg table evolution (schema evolution, partition spec, snapshot management)
• Develop best practices for Iceberg governance, metadata compaction, and performance tuning

• Build scalable batch and streaming pipelines using AWS services (S3, EMR, Glue, Lambda, Step Functions)
• Develop ingestion and transformation workflows using Python, Spark, or Flink
• Implement CDC pipelines using Kafka Connect or equivalent tooling
• Ensure robust CI/CD integration with GitHub Actions or similar

• Design and operate Kafka-based streaming pipelines (Kafka/MSK)
• Build producers/consumers using Python or JVM languages
• Implement patterns such as topic partitioning, compaction, schema registry, and event versioning

• Design data models for analytical and operational use cases using Iceberg tables
• Implement automated data quality checks, validation rules, and anomaly detection
• Build lineage, monitoring, alerting, and pipeline observability

• Apply best practices for AWS security, cost optimization, and data governance
• Manage IAM, KMS, S3 object lifecycle management, networking, and data encryption
• Operationalize EMR/Glue jobs, containerized workloads, or serverless workloads

• Partner with analytics, platform, and product teams to deliver high-quality data products
• Participate in design reviews, architecture discussions, and roadmap planning
• Mentor junior engineers and contribute to engineering standards
• 4-10+ years of experience in Data Engineering or similar roles
• Strong hands-on experience with Apache Iceberg (table design, evolution, metadata, partitioning)
• Deep experience with AWS data stack: S3, EMR, Lambda, Glue, IAM, Step Functions, CloudWatch
• Strong proficiency in Kafka (producers/consumers, schema registry, partitioning strategies)
• Fluency in Python for data pipelines, automation, and APIs
• Experience with distributed engines such as Spark, Flink, or PySpark
• Expertise in scalable ETL/ELT pipelines and real-time streaming architectures
• Strong SQL and data modeling expertise