Title: Principal Data Engineer

Department: Technology

Salary: $110K - $120K CAD

Location: Remote (Candidates based in British Columbia, Alberta, Manitoba, Saskatchewan, Ontario, Nova Scotia or willing to relocate)

Description:

The Principal Data Engineer will be responsible for designing, developing, and maintaining our data architecture, with a focus on Snowflake and ETL processes. The ideal candidate will have a deep background in data engineering, expertise with Snowflake's cloud data platform, and highly proficient in ETL tools and methodologies. As part of our data team, you will play a crucial role in elevating our data infrastructure, ensuring scalability, reliability, and efficiency in handling large volumes of data.

Duties included but not limited to:

• Integrate diverse data sources by designing and implementing scalable high performance data pipelines, architecture and infrastructure.
• Build and implement ETL processes using a variety of methods to facilitate efficient movement of data between systems
• Develop culture to incorporate best practices to ensure data pipelines are robust and include error handling and alerting
• Design data models that turn complex organizational data into easy to use data structures ensuring seamless flow for WES’S operational, analytics and reporting needs
• Use DBT or Snowflake native capabilities to transform data and ensure data is accurate and up to date
• Optimize data structures and queries to ensure efficient data storage and retrieval.
• Implement and enforce data quality standards to ensure the accuracy and reliability of educational data used in academic assessments globally.
• Collaborate with cross-functional teams, including analysts and business stakeholders, to understand unique data requirements for data processing or analytics needs
• Implement and maintain data security measures to protect sensitive information, ensuring compliance with international data privacy regulations.
• Develop and maintain practices for effective data warehousing, transformations, data sharing, and structured management
• Create and maintain comprehensive documentation for data engineering processes, systems, and workflows
• Provide technical leadership and mentorship to junior members of the data engineering team, fostering a collaborative environment dedicated to excellence in handling educational data
• Utilize Tableau to deliver data insights using impactful visualizations in a meaningful and actionable manner

Requirements:

• Education: in Computer Science, Information Technology, or a related field, with a strong emphasis on Data related technologies
• 8+ years’ experience in a data engineering role, with a focus on designing and implementing scalable data solutions
• Highly proficient in programming languages such as Python, Java, or Scala.
• Expertise in working with both SQL and NoSQL databases, with hands-on experience in database design and optimization. Proficiency in Snowflake is required
• Expertise in Snowflake for effective data warehousing, transformations, data sharing, and structured management in a cloud-based environment
• Well versed in Machine Learning concepts and commitment to growth in this area.
• Proficient with cloud platforms like Azure, AWS, or Google Cloud Platform
• Experienced using Azure for Data Engineering tasks i.e. function apps and other Azure functionality that can be used to build data pipelines a plus
• Knowledge of version control systems like Git or Azure DevOps for managing the codebase
• Strong analytical and problem-solving skills to address complex data engineering challenges specific to educational data
• Excellent communication skills to effectively collaborate with cross-functional teams and non-technical stakeholders
• Ability to adapt to evolving technologies and industry best practices in the field of data engineering
• Experience using TIBCO EBX a plus
• Experience using Boomi or other low code ETL tools a plus