Lead Data Engineer

Job Title: Lead Data Engineer Location: Mississauga / Hybrid Employment Type: Full-time Role Overvie wWe are seeking an experienced Data Engineer with strong expertise in large-scale data processing and cloud-based application development. The role requires deep hands-on technical skills along with a strong understanding of end-to-end data solutions and the ability to collaborate across teams to deliver scalable and high-performance systems . Experience Requiremen • ts8+ years of experience in software development, preferably in distributed data systems or large-scale applicatio • nsMinimum 5+ years of hands-on experience in designing and developing Big Data solutio • nsAt least one end-to-end Big Data implementation experien • ceKey Responsibiliti • esDesign, develop, and maintain scalable Big Data pipelines and data processing syste • msBuild distributed data processing applications using modern framewor • ksDevelop backend services and APIs for data platfor • msOptimize performance and troubleshoot issues in large-scale data syste • msWork closely with cross-functional teams to deliver end-to-end solutio • nsEnsure data quality, scalability, and reliability of data pipelin es Technical Ski • llsProcessing & Comp • uteStrong hands-on experience with Apache Spark (mandato • ry)Experience with Hadoop ecosystem including HDFS, Hive, Impala, HB ase Program • mingProficiency in Java / Scala / Py • thonData & Q • ueryStrong expertise in SQL (query building, debugging, optimizat • ion)Data For • matsExperience working with XML, JSON, Parquet, Avro, Prot • obufApplication Develop • mentExperience in Spring Boot / Microservices architec • tureStrong experience in backend development and API de • signMessaging & Strea • mingHands-on experience with Apache Kafka for real-time data pipel • inesAI & Productivity T • oolsAbility to leverage tools such as GitHub Copilot, ChatGPT, or similar to enhance development efficiency, code quality, and debug • gingNice-to-Have Sk • illsExperience with Databricks Lakehouse Platform, includ • ing:Delta • LakeDatabricks Workf • lowsUnity Cat • alogML • flowSoft Sk • illsStrong analytical and troubleshooting capabili • tiesAbility to work under pressure in a fast-paced environ • mentStrong multitasking and prioritization sk • illsHigh learning agility and proactive min • dsetAdaptable, flexible, and energetic approach to problem-sol ving Karat assessment is mandatory for all candid ates.

Want to see how well you match this job?