Azure Cloud Data Engineer-11324-Remote at Shuvel Digital #vacancy #remote

Basic Purpose:

Develop technical solutions for data acquisition, data integration, and data sharing with Navy Federal’s digital platforms and fraud business in Near Real-Time and batch. Responsible for engineering, designing, building, integrating data from various batch, streaming, edge applications into high performing operational Hubs. Solves highly complex problems; takes a broad perspective to identify solutions.

Responsibilities:

  • Design and build highly scalable data pipelines for near real-time and batch data ingestion, processing, and data integration.
  • Technical leadership and knowledge to provide technical guidance and educate team members and coworkers on development and operations of streaming and event driven applications.
  • Recognize potential issues and risks during the project implementation and suggest mitigation strategies.
  • Communicate and own the process of manipulating and merging large datasets.
  • Work directly with business leadership to understand data requirements; propose and develop solutions that enable effective decision-making and drives business objectives.
  • Perform other duties as assigned

Qualifications and Education Requirements:

  • Degree in Information Systems, Computer Science, Engineering, or related field, or the equivalent combination of education, training, and experience
  • Working knowledge of message-oriented middleware/streaming data technologies such as Kafka, MQ, Azure Event Hub, Spark, Spark Streaming
  • Must have strong programming skills / experience in C# / .NET, Logic App
  • Must have strong programming skills / experience in Azure Functions using various protocols /triggers.
  • Hands-on experience in configuring Azure Event Hub, Event Grid, Stream Analytics, logic/function apps and JSON.
  • Expert-level skills in Python, Databricks, Azure Data Factory
  • Experienced in the use of ETL tools and techniques and have knowledge of CI/CD
  • Experience & Expertise in cloud NoSQL databases, ideally Cosmos DB or equivalent

azure-cosmosdb databricks data-sharing Streaming media Apache Spark Computer Science Apache Kafka JSON data-pipelines Azure Functions Engineering C# Software Developer data-acquisition Azure Data Factory spark-streaming CI/CD Python Azure Data Engineer Big data data-integration Azure Stream Analytics Azure EventHubs Information Systems ETL .NET batch-file NoSQL Leadership

Залишити відповідь