Sr. Data Engineer Remote – Onsite 3 days per month in Greater Chicago Position Summary: This role is responsible for providing technical expertise and leadership to design and deliver end-to-end data engineering solutions to support advanced analytics capabilities and drive innovation and decision-making across our enterprise. Responsibilities: Build and maintain real-time and batch data pipelines across the advanced analytics platform. Design, develop and orchestrate highly robust and scalable ETL pipelines. Design and implement Dimensional and NoSQL data modeling as per the business requirements. Develop highly optimal codebase and perform Spark optimizations for Big Data use cases. Design, develop and deploy optimal monitoring and testing strategy for the data products. Collaborate with stakeholders and advanced analytics business partners to understand business needs and translate requirements into scalable data engineering solutions. Collaborate with data scientists to prepare data for model development and production. Collaborate with data visualization and reporting application developers to ensure the sustainability of production applications and reports. Collaborate with data architects on the enhancement of our enterprise data architecture and platforms. Provide leadership to third-party contractors. Comply with health and safety guidelines and rules. Maintain professional and technical knowledge by attending educational workshops, professional publications, establishing personal networks, and participating in professional societies. Requirements: 4+ years of experience in Data Engineering 2+ years of experience with Advanced Analytics or Data Science Proficient in Spark Experience with Azure Nice to have: Experience with IoT Data Architecture. Machine Learning Operationalization (MLOps) proficiency. Proficiency with streaming design patterns
Innovativeness Data Science Apache Spark testing Data Engineering Azure Publications Technology savvy End-to-end testing data-visualization Big data data-modeling MLOps Enterprise Data Architecture Decision-making Analytics Design Systems monitoring ETL batch-file Leadership NoSQL