As a Lead Data Engineer at Brown and Caldwell, you will play a pivotal role in designing, building, and maintaining robust data pipelines and architectures that enable effective data integration, storage, and analysis. We are using modern technology to transform the way that water is managed across the country. Your work will directly impact our ability to build digital solutions that ensure a future of clean water, thriving communities, and environmental protection. Collaborate with interdisciplinary teams of environmental engineers, data scientists, and software developers to understand data requirements and develop scalable data solutions. Design, implement, and maintain data pipelines to collect, clean, transform, and load data from various sources, including sensor data, historical records, and geospatial information. Create and manage data warehouses, data lakes, and databases to ensure efficient data storage, retrieval, and management. Develop and deploy ETL (Extract, Transform, Load) processes to support data analysis, visualization, and machine learning model training. Implement data quality and data governance processes to ensure accuracy, consistency, and integrity of data. Collaborate with IT teams to optimize data infrastructure, performance, and security. Stay up to date with emerging technologies, tools, and best practices in data engineering and environmental engineering to drive continuous improvement. Desired Skills and Experience: Bachelor’s or Master’s degree in Computer Science, Data Engineering, Data Science, Engineering, or a related field with a focus on data engineering, database management, or information systems. 8+ years of experience in data engineering, database design, and ETL development in a professional setting. High proficiency in programming languages such as SQL, Python, Java, or Scala, with experience in data processing system such as Apache Spark, Synapse Data Engineering in Microsoft Fabric, etc. Demonstrated knowledge of software engineering principles, version control, and best practices for writing clean, maintainable, and scalable code including development of custom GitHub Actions and Workflows for CI/CD. Familiarity with related data integration tools, data orchestration frameworks, and version control system. Solid understanding of database concepts, data modeling, and query optimization for both structured and unstructured databases. Experience with data warehousing solutions (e.g., Azure Synapse Analytics, Amazon Redshift), data lake solutions, and cloud platforms (e.g., Azure, AWS). Experience developing functional data processing workflows to transform raw data into reliable input for ML algorithms. Experience with streaming data pipeline setup and processing in IoT and edge compute applications. Hands-on experience with PySpark. Experience with Azure platform, DevSecOps and MLOps, specifically the following: Advanced knowledge of current Microsoft Azure products, including: Azure Machine Learning Studio Azure Data Factory Azure Databricks Azure Synapse Azure Monitor and Azure EventGrid Azure Cognitive Services Knowledge of geospatial data processing and analysis is a plus. Strong problem-solving skills and the ability to work in a collaborative, cross-functional environment. Excellent communication skills to interact with technical and non-technical stakeholders. A passion for staying updated with the latest trends, tools, and technologies in data engineering and environmental engineering. Salary Range: The anticipated starting pay range for this position is based on the employee’s primary work location and may be more or less depending upon skills, experience, and education. These ranges may be modified in the future. Location A: $123,000 – $169,000 Location B: $136,000 – $185,000 Location C: $148,000 – $202,000 You can view which BC location applies to you here. If you have any questions, please speak with your Recruiter. Benefits and Other Compensation: We provide a comprehensive benefits package that promotes employee health, performance, and success which includes medical, dental, vision, short and long-term disability, life insurance, an employee assistance program, paid time off and parental leave, paid holidays, 401(k) retirement savings plan with employer match, performance-based bonus eligibility, employee referral bonuses, tuition reimbursement, pet insurance and long-term care insurance. Click here to see our full list of benefits. About Brown and Caldwell Headquartered in Walnut Creek, California, Brown and Caldwell is a full-service environmental engineering and construction services firm with 52 offices and over 1,900 professionals across North America and the Pacific. For more than 75 years, we have created leading-edge environmental solutions for municipalities, private industry, and government agencies. We strive to be the company of choice—to our clients, who benefit from our passion for delivering exceptional quality, and to our employees, present and future, who share our commitment to client service, collaboration, and innovation. Join us, and you will find a home where you can do your best work, reach new levels of expertise, and enjoy exceptional development opportunities. For more information, visit This position is subject to a pre-employment background check and a pre-employment drug test. Notice to Third Party Agencies: Brown and Caldwell does not accept unsolicited resumes from recruiters or employment agencies. In the event a recruiter or agency submits a resume or candidate without a previously signed agreement and approved engagement request with Brown and Caldwell, Brown and Caldwell reserves the right to pursue and hire those candidate(s) without any financial obligation to the recruiter or agency. Brown and Caldwell is proud to be an EEO/AAP Employer. Minorities/Women/Disabled and Protected Veterans are encouraged to apply. Brown and Caldwell ensures nondiscrimination in all programs and activities in accordance with Title VI of the Civil Rights Act of 1964. Let us know your job expectations, so we can find you jobs better! Senior Program Manager, Cybersecurity Compliance Full Time Atlanta, Georgia Atlanta, Georgia Atlanta, Georgia Atlanta, Georgia Pleasanton, CA Pleasanton, CA Pleasanton, CA Pleasanton, CA Salt Lake City Salt Lake City Salt Lake City Salt Lake City United States United States United States United States Utah Utah Utah Utah Workday Workday Full Time Full Time Atlanta, Georgia , Pleasanton, CA , Salt Lake City , United States , Utah Send to friend Share Distributed Systems Engineer Full Time United States United States United States United States Ai+ Career Services Ai+ Career Services Full Time Full Time United States Send to friend Share Senior Data Scientist Full Time New York New York New York New York United States United States United States United States Ai+ Career Services Ai+ Career Services Full Time Full Time New York , United States Send to friend Share Full Address 201 N Civic Dr, Walnut Creek, California 94596, US Connect with us Contact Us Here at the Open Data Science Conference we gather the attendees, presenters, and companies that are working on shaping the present and future of AI and data science. ODSC hosts one of the largest gatherings of professional data scientists with major conferences in the USA, Europe, and Asia. Please let us know who you are to finish the registration I’m an employer looking to hire I’m a candidate looking for a job #J-18808-Ljbffr
azure-monitor Apache Spark Computer Science data-security Data Engineering database-management Machine-learning algorithms Communication data-modeling MLOps data governance version-control IoT Engineering azure-databricks data-quality PySpark Azure Data Factory state-of-the-art technologies Scala query-optimization Python Data Science Problem-solving data-warehouse azure-synapse Amazon Redshift cloud-platforms GitHub Actions Software Development Engineer database-theory devsecops data-integration SQL Information Systems Java azure-cognitive-services ETL