Data Warehouse System Engineer

Hyve Company Overview Hyve Solutions transforms complex engineering challenges into production reality for technology innovators building AI, cloud, and connected infrastructure. As a US-based manufacturing partner, the company rapidly delivers fast, agile execution through dep technical partnerships and co-innovation, and supply chain clarity. Hyve’s integrated ODM, CM, and SI capabilities eliminate vendor complexity while accelerating time-to-market with single-partner accountability from design through scale. The company co-innovates with deep engineering expertise, treating customer success as its own while building tomorrow's digital infrastructure. Job Title Data Warehouse SE Job Description We are seeking an experienced Data Warehouse SE (Data Platform Engineer) to lead the evolution of our enterprise data platform from a traditional Hadoop/Hive architecture to a modern cloud-native Lakehouse platform built on Apache Iceberg and object storage. This role focuses on platform engineering, building and operating large-scale data infrastructure rather than developing business-specific data pipelines. The ideal candidate will have deep expertise in distributed data systems, cloud-native architectures, and large-scale data processing technologies. You will be responsible for designing and delivering core platform capabilities including: Compute-storage separation architectures Apache Iceberg-based data lake implementations Unified access frameworks across multiple processing engines (Spark, Flink, Trino, etc.) Reliable operation of data workloads on Kubernetes and object storage Solving large-scale engineering challenges related to performance, scalability, concurrency, cost optimization, and operational stability at PB-scale Key Responsibilities: Design, build, and operate an enterprise-scale Lakehouse platform based on Apache Iceberg to support batch, streaming, and incremental data workloads. Drive the migration from Hadoop/HDFS to cloud-native object storage architectures and implement compute-storage separation at scale. Develop and enhance core platform services, including metadata management, catalog systems, governance frameworks, and multi-engine data access capabilities. Optimize distributed processing engines such as Spark to improve performance, scalability, reliability, and cost efficiency in PB-scale environments. Build and operate cloud-native data platforms on Kubernetes, including workload orchestration, resource management, and operational automation. Integrate the platform with cloud ecosystems such as Azure, Databricks, and Snowflake to enable seamless data flow and interoperability. Develop internal platform components and engineering tools for data ingestion, orchestration, quality management, diagnostics, and AI-driven automation. Required Skills Expert-level knowledge of Apache Spark, including execution planning, shuffle processing, memory management, and performance optimization. Strong experience with Apache Iceberg, Hudi, or Delta Lake, including metadata architecture, snapshots, manifests, schema evolution, and partition evolution. Hands-on experience deploying and managing data workloads on Kubernetes, including Spark on Kubernetes, Operators, and resource scheduling. Solid understanding of cloud object storage platforms such as Azure Blob Storage, Amazon S3, or MinIO, including storage consistency and performance characteristics. Deep understanding of modern Lakehouse architecture, including Iceberg, Spark, Catalog services, and object storage ecosystems. Experience with at least one cloud data platform such as Azure, Databricks, or Snowflake. Strong software engineering and programming skills in Java, Scala, or Python. Preferred Qualifications Experience designing and delivering enterprise-level data platform solutions rather than single business applications. Hands-on experience with Lakehouse implementation, data lake modernization, or platform migration initiatives. Proven experience deployi…

View the full posting and apply