Salary Dependent on Experience
Closing Date for this application is 01/08/2020
If this position receives high volumes of applications we reserve the right to close this advert earlier than stated, so please apply early to avoid disappointment.
We are seeking a Data Engineer to join us in our growing Data Infrastructure & Engineering team; to help develop, maintain, support, and integrate our growing number of data systems. The right candidate will have a wealth of knowledge in the data world and will be keen to expand upon their existing knowledge set learning new technologies along the way as well as supporting legacy technologies and processes.
In this role, you will:
Ensure that data pipelines are scalable, repeatable, and secure, and can serve multiple users and applications within the Group
Help facilitate getting data from a variety of different sources, getting it into the right formats, assuring that it adheres to data quality standards, and assuring that downstream users can get that data quickly.
Enable big data and batch / real-time analytical solutions that leverage emerging technologies
Develop prototypes and proof of concepts for selected solutions
Implement complex big data projects with a focus on collecting, parsing, managing, analysing, and visualising large sets of data using multiple platforms
You would be responsible for:
Coding, testing, and documenting new or modified data systems; creating scalable, repeatable, secure pipelines and applications for data analytics, both internally and externally to the business
Growing the capabilities of the data platform(s), solving new data problems and challenges
Supporting all manner of data flows for batch/real-time analytical solutions leveraging transformational technologies
Maintaining the infrastructure that provides insight from raw data and handles diverse sources of data seamlessly
Ensuring that all automated processes preserve data by managing the alignment of data availability and integration processes
Designing key and indexing schemas and designs partitioning
Assisting with the development, testing, implementation, monitoring and maintenance of 24x7 production database management systems
Participating in building and testing business continuity and disaster recovery procedures as per requirements
Ensuring that all source data systems in the production environment remain stable and usable
Required education and experience:
A minimum 5 years’ experience software development or database experience
Proficiency in writing intermediate-level SQL including stored procedures in languages such as T-SQL. PL/PgSQL and PL/SQL
Proficiency in one or more database scripting languages (i.e., PowerShell, Python etc.,)
Experience with relational database internals, including both query processing and query planning, or other data processing infrastructure across several different RDBMS systems as well as NoSQL systems
A basic knowledge of key data structures and algorithms
Capability in programming in 3GL languages such as C#, C, C++ or Java
A solid understanding of physical database design principles, and the system development life cycle.
Understand how to apply technologies to solve data problems and to develop innovative data related solutions
Knowledge of data analysis, end-user requirements analysis, and business requirements analysis to develop a clear understanding of the business needs and to incorporate these needs into technical solution
Experience with object-oriented design, coding, and testing patterns
Extensive experience in Microsoft SSIS, specifically designing and implementing ETL packages
Knowledge of data modelling and understanding of different data structures and their benefits and limitations
Familiarity with monitoring, backup, and disaster recovery of data systems
Experience in designing, architecting, and building scalable pipelines
Extensive experience with Linux, Unix and Microsoft platforms.
Desirable skills & Technologies
Knowledge and usage of BI Tools such as Looker, Power BI, Tableau
Experience with Spark and other analytical programming languages
Experience in utilising Microsoft Azure and any related tooling for data engineering (such as Azure Data Warehouse, Azure Data Lake, EventHub, Apache Spark, Azure Database, Azure Databricks, Synapse)
Experience on engineering big data systems
A solid working knowledge of database administration methodologies and techniques
A good understanding of logical and physical database design principles.
Strong communication skills at all levels
Be a natural problem solver
Work well in both functional and cross-functional environments
Exhibit flexibility when addressing daily shifting work priorities (we are a 24x7 business)
Have the ability to work well under pressure, using own initiative as well as part of the wider organisation
If you believe you meet the above criteria and feel that this could be the role for you then APPLY NOW!