Data Engineer

Job DescriptionInfiligence is a multi national AI-led Platform Engineering company with offices in Chennai, Calgary, and Pleasanton, California. We are committed to delivering innovative AI solutions to our clients worldwide. We offer a collaborative work environment, competitive compensation, and comprehensive employee benefits. Key Responsibilities Design, develop, and maintain scalable data pipelines for batch and real-time data processing using Azure (preferred) or AWS cloud platforms. Build and optimize ETL processes to ensure high-quality, secure, and efficient data flow across systems. Collaborate with cross-functional teams to translate business requirements into robust data models and solutions. Implement data quality, data governance, and data security standards throughout the data lifecycle. Develop and maintain documentation, including design and mapping documents. Conduct code reviews, unit testing, and peer reviews to ensure code quality and compliance. Troubleshoot, monitor and resolve data pipeline and infrastructure issues to minimize business impact. Stay updated with new technologies and evaluate their organizational impact. Required Qualifications 5–7 years of experience in data engineering, with a strong background in building and managing large-scale data pipelines. Hands-on experience with Azure Data Services (Data Factory, Data Lake, Synapse, Databricks, etc.) is preferred. Experience with AWS data services is also acceptable. Proficiency in SQL, Python, or Scala for data processing and transformation. Experience with data warehousing (e.g., Snowflake, SQL Server, MongoDB) and real-time databases. Strong understanding of data architecture, data ingestion, curation, and consumption patterns. Familiarity with data quality management, metadata management, data lineage, and data security best practices. Excellent communication skills and ability to work collaboratively with global teams. Preferred Skills Experience with CI/CD processes and source control for data engineering workflows. Knowledge of data observability and self-testing pipelines. Exposure to business intelligence and reporting platforms. 

Apply Now