Job Title

Responsibilities: Design & Architecture of Scalable Data Platforms Design, develop, and maintain large-scale data processing architectures (Lakehouse Platform to support business needs) Architect multi-layer data models including Bronze (raw), Silver (cleansed), and Gold (curated) layers for HRTech Domains Strong experience with any of these Snowflake, Databricks, Redshift. Leverage Delta Lake, Unity Catalog, and advanced features of Data-bricks for governed data sharing, versioning, and reproducibility. MUST have experience with AWS technologies like AWS glue, Athena, Redshift, etc. Data Pipeline Development & Collaboration Collaborate with data engineers and data scientists to develop end-to-end pipelines using Python, PySpark, SQL Performance, Scalability, and Reliability Optimize Spark jobs for performance tuning, cost efficiency, and scalability by configuring appropriate cluster sizing, caching, and query optimization techniques. Implement monitoring and alerting using Observability Platforms, Cloud-native tools Design secure architectures using Unity Catalog, role-based access control (RBAC), encryption, token-based access, and data lineage tools to meet compliance policies. Establish data governance practices including Data Fitness Index, Quality Scores, SLA Monitoring, and Metadata Cataloging. Writing PySpark, SQL, and Python code snippets for data engineering and ML tasks. Data profiling and schema inference. Stay abreast of emerging trends in Lakehouse architectures, Generative AI, and cloud-native tooling. Requirements: 8-12 years of hands-on experience in data engineering, with at least 5+ years on Python and Apache Spark. MUST have experience in migrating to data lakes from RDBMS like systems. Expertise in building high-throughput, low-latency ETL/ELT pipelines on AWS using Python, PySpark, SQL, Athena, AWS glue, Redshift, etc Excellent hands on experience with workload automation tools such as Airflow, Prefect etc. Familiarity with building dynamic ingestion frameworks from structured/unstructured data sources including APIs, flat files, RDBMS, and cloud storage Experience designing Lakehouse architectures with bronze, silver, gold layering. Strong understanding of data modelling concepts, star/snowflake schemas, dimensional modelling, and modern cloud-based data warehousing. Experience with designing Data marts using Cloud data warehouses and integrating with BI tools (Power BI, Tableau, etc.). Experience CI/CD pipelines using tools such as AWS Code commit, Azure DevOps, GitHub Actions. Knowledge of infrastructure-as-code (Terraform, ARM templates) for provisioning platform resources In-depth experience with AWS Cloud services such as Glue, S3, Redshift etc. Strong understanding of data privacy, access controls, and governance best practices. Experience working with RBAC, tokenization, and data classification frameworks