Senior Data Engineer

 Role overview

An exciting data business is seeking a dynamic and entrepreneurial individual to operate its consumer data environment and service its data requirements to deliver on the business’ client and operational needs. The role will enable the individual to become the owner of a complex and exciting data environment within a bigger group of data businesses.

They would need to work closely with the client management team, data scientists, analysts, and other stakeholders to understand data requirements and ensure the availability, quality, and accessibility of the data

Being a young company that is part of a group that is mature and experienced in data management, they will also need to work closely with the group data function to provide input and seek guidance to enable a consistent approach to how data is managed across the group and form part of the larger group data function

The role will have a direct reporting line into the Business Unit (MD/COO) on guidance on work and prioritization and a dotted reporting line into the Group Head of Data on guidance on technical and architectural guidelines.

 Duties

Key responsibilities would include:

Design and own all aspects of data engineering from delivery planning, estimating and analysis all the way through to data architecture and pipeline design, delivery, and production implementation. Construct end to end data service solutions.

Data Pipeline Development: Designing, implementing, and optimizing scalable and efficient data pipelines to extract, transform, and load (ETL) data from various sources into data storage systems. This may also require engagement and collaboration with other data teams from whom data is sourced.

Data Integration: Integrating data sources and systems, ensuring seamless data flow and compatibility between different environments. Design and implementation of complex data solutions ranging from batch to hybrid landscapes. Employ a variety of languages and tools (e.g. scripting languages) to marry systems together.

Data Quality Assurance: Implementing data quality checks, data validation processes, and data cleansing techniques to ensure the accuracy, consistency, and integrity of data.

Data Monitoring and Troubleshooting: Proactively monitoring data processes and systems, identifying and resolving data-related issues, and optimizing performance bottlenecks. Recommend ways to improve data reliability, efficiency and quality.

Operating and Maintaining Data Tools: Assess and improve data tools that have been built within the business and operate these data tools as and when needed. Automate data processes end to end (acquire, ingest, transform, model, store)

Collaboration and Documentation: Collaborating with cross-functional teams, such as data scientists, analysts, and software engineers, to understand data requirements and deliver efficient data solutions. Documenting data processes and configurations for knowledge sharing and future reference. Contribute towards a team that develops, constructs, tests and maintains architectures (such as databases and processing systems) and ensure data architecture will support the stakeholder requirements. Liaise and interface with stakeholders in technical advisory to support and enhancement services.

Emerging Technologies: Staying updated with industry trends and emerging technologies related to data engineering, recommending innovative solutions to improve data management and analytics capabilities.

Working Model

Hybrid working model – with a requirement to be in office in Johannesburg/Centurion as required twice per week (1 day in Centurion and 1 day in Bryanston)

Qualifications and Experience

•                 Matric (Grade 12)

•                 Applicable Diploma or Degree

•                 5 to 10 years working in/across Data Engineering, Analysis & Data Warehousing

•                 At least 5 years’ experience with SQL and SSIS packages

Technical skills

•                 Proficient in Microsoft SQL Server

•                 Strong proficiency in TSQL programming (stored procedures, user defined functions, queries, triggers)

•                 Constructing data queries by combining multiple data sources.

•                 Knowledge of data integration techniques and tools, such SSIS for ETL/ELT processes, APIs, and data streaming an advantage

•                 Data Cleaning / Integrity Checking

•                 Proficiency in data modeling and database design of relational databases.

•                 Iterative Testing including debugging and refactoring.

•                 Understanding of data governance, security, and privacy principles.

•                 Experience working in a multi-disciplinary team where your own responsibility included transforming data into clean, organized data structures with the emphasis of design supporting analysis and construction of data products

•                 Ability to be self-sufficient from a technical point of view

•                 Experience of creating reports using Excel or equivalent.

•                 Experience with data visualization in Power BI and similar tools

•                 It will also be an advantage to have:

o   Some experience in a programming language (such as Python/R).

o   Data Architecture experience

o   Data Management and Governance

o   Experience of consuming APIs

o   Data Warehousing automation

o   Experience with consumer and retail data

Non-technical skills and competencies

·        Strong problem-solving and analytical skills, with attention to detail. (Core to the role)

·        Excellent communication and teamwork abilities, especially with the ability to effectively communicate with non-technical people

·        Enjoy supporting technical implementations.

·        Able to build strong networks and relationships, internally and with customers.

·        Can effectively multi-task and manage conflicting priorities, coupled with good attention to detail.

·        Aptitude for providing timely and accurate responses to client inquiries and managing client expectations.

·        A passion to learn and extend knowledge outside of the work sphere.

·        Capable of identifying, embracing and initiating change in an agile and fast-faced environment.

·        The individual must display the characteristics of a self-starter. As such be an individual who takes initiative and demonstrates a proactive and motivated approach to their work. They should be driven by intrinsic motivation and must not rely heavily on external direction or supervision to get things done. Key characteristics include:

o   Initiative: should be someone who takes the initiative to identify and pursue opportunities or tasks without needing to be explicitly told or instructed.

o   Motivation: Should be internally motivated and have a strong drive to achieve their goals and deliver results. They set high standards for themselves and constantly seek self-improvement.

o   Resourcefulness: Should be a resourceful individual who is able to find solutions to problems independently. Should be proactive in seeking out information, tools, and resources to overcome challenges and achieve objectives.

o   Independence: Can work effectively with minimal supervision or guidance. They are self-reliant and have the ability to make decisions and take responsibility for their actions.

o   Proactivity: Be proactive in identifying opportunities, anticipating potential issues, and taking action to address them.

o   Time Management: Should have good time management skills and capable of prioritizing tasks and managing workload effectively. Should be capable to set goals, create plans, and meet deadlines without constant reminders or supervision.

o   Adaptability: Be adaptable and flexible in approach. Quickly adjust to changing circumstances, learn new skills, and embrace challenges with a positive mindset.

o   Persistence: Should demonstrate perseverance and determination when faced with obstacles or setbacks. They maintain a positive attitude and remain focused on their goals, finding alternative solutions if needed.

·        Overall, be proactive and motivated individual who takes ownership of their work, initiates action, and strives for success without extensive external guidance or supervision.

·        The Individual will be expected to operate in line with our company values:

o   Passion: We have a strong and barely controllable excitement for life, our work, the people we work with, and for ourselves. Our enthusiasm and positive energy lead to a strong bias toward action and strive for excellence!

o   Care: We care deeply for ourselves, the people we work with, our customers, the communities we live in, the country we call home and the planet which hosts us!

o   Integrity: We always do the right thing, even when nobody is watching!

o   Growth: We always strive to make a positive contribution on our journey to significance! We (us and the business) grow, because our people grow! Our curiosity drives Innovation and our resilience keeps us determined when the going gets tough! We have the courage to make decisions and to have the tough conversations.

o   Diversity: We respect and cherish each other and listen to all opinions! We are all unique, which makes the “us” stronger! Through collaboration, we ensure that the whole is greater than the parts.

What we offer

·        The opportunity to work with an experienced data team across the group and within a group of proven successful data business in South Africa.

·        An opportunity to be responsible for a small data operations unit

·        To be part of a South African group that has expanded globally

·        A leadership team that focusses on a care and growth style to leadership and cares deeply for their people

·        An opportunity to be part of a cutting edge data business

·        Hybrid working model