Azure Data and BI Engineer

In this role, you will design, develop, modify, adapt, and implement both short- and long-term data and information technology solutions through new and existing applications, systems, databases, and infrastructure. You will review and interpret system requirements and business processes. You will code, test, debug, and implement software solutions, ensuring the seamless integration and security of data pipelines. This role requires the design, development, and implementation of Data and Business Intelligence (BI) solutions to business problems, primarily utilizing Azure services. The ability to adapt and learn new technologies per business requirements is crucial. Duties encompass the entire data lifecycle, from ingesting, transforming, and loading data of various sources, volumes, formats, and velocities to meet business needs. This also includes leveraging the processed data to generate meaningful insights through BI tools, creating compelling data visualizations, and ultimately, supporting decision-making processes by providing insightful business analytics.

Responsibilities

• Design and implement effective data solutions utilizing industry best practices.
• Refine and perform ETL/ELT operations using tools like PySpark to ensure secure administration of data and systems, compliance with enterprise data governance standards, and optimized resource usage for maximum data processing speed.
• Proactively monitor data pipelines to ensure high service availability.
• Design and maintain efficient data architectures using best practices for normalization, indexing, and partitioning.
• Develop scalable data pipelines for structured and unstructured data using Azure Data Factory, Azure Databricks, or similar tools.
• Develop and implement data quality checks to ensure data reliability and accuracy.
• Create and maintain interactive dashboards and data visualizations using tools like SSRS, Tableau, Power BI, and Looker to enhance decision-making processes.
• Design and manage data integration and reporting workflows using SSIS, SSRS, for data processing and transformation.
• Design and execute complex queries and reports, presenting data insights to stakeholders in a user-friendly format.
• Collaborate with teams to understand data needs and implement BI solutions that ensure accuracy and consistency.
• Document data architecture, definitions, charts, and reports to ensure data integrity and facilitate efficient usage.
• Develop efficient data models and database structures that optimize storage and retrieval.
• Ensure data privacy and security compliance when developing BI and data solutions.
• Work with data scientists, machine learning engineers, and business analysts to optimize data solutions based on mathematical models and actionable pipelines.
• Participate in continuous development through training and mentorship programs.
• Create scripts and programs to automate data operations.

Qualifications

• Minimum bachelor’s degree in data science, Business Intelligence, Computer Science, or related fields, or the equivalent combination of education, professional training, and work experience.
• 0-3 years of experience working with one or more programming languages commonly used for data operations, including SQL, Python, Scala, and R.
• Experience working with relational databases such as SQL Server, Oracle, and MySQL.
• Experience working with NoSQL databases such as Redis, MongoDB, and Cosmos DB.
• Strong problem-solving skills and the ability to learn through scattered resources.
• In-depth understanding of the responsibilities and duties of a Data and BI Engineer, as well as established industry standards/best practices and documentation guidelines.
• Outstanding communication skills, with the ability to stay self-motivated and work with minimal supervision.
• Authorization(s) to work lawfully in the United States.