Hadoop Systems Engineer

Hadoop Systems Engineer

Hadoop Systems Engineer

Location: London

Salary: Competitive

Expiry: 25/02/18

Why is our Data Transformation team the next step for you?

Worldpay is one of the largest global payments (Fintech) businesses in the world that has put data at the centre of the business by creating a new Data Transformation team that spans business and technology. At the heart of this is our large Hadoop based Enterprise Data Platform that is used to provide data insights both to our internal teams and our customers. The Enterprise Data Platform (EDP) is an innovative use of “Big Data” technologies to provide powerful analytic capabilities for Worldpay. For this, we currently use two Hadoop clusters in a HA and DR configuration with an expansion plan to many hundred nodes; based on the Hortonworks Data Platform.


Your key responsibility will be to ensure that the Hadoop platform at the heart of this initiative is secure, available, performant and fit for purpose.


This is a great opportunity to expand your existing skills in the Hortonworks Data Platform and the related tools in an innovative and passionate environment.

How will you add value on a day-to-day basis?

You will be a key contributor to the Hadoop development operation and administration of the Enterprise Data Platforms Hadoop Clusters while working in collaboration with other team members. It would include maintenance of a mission critical HA and DR solution. You will be involved in the end-to-end performance tuning of Hadoop clusters and Hadoop Map/Reduce routines against very large data sets. Also monitoring and managing Hadoop cluster performance and capacity planning. You will be implementing and managing additional nodes on the Hadoop cluster including its connectivity and security within a highly secure environment (Knox, Ranger, Kerberos, Active Directory, TLS and SPNEGO).

You will be conducting Hadoop cluster training and on the other side, involved in 3rd party software installation and configuration. You will also be the Point of Contact for Vendor escalation. This is a new cluster and provides the successful application, a chance, to help define the breadth of responsibility and future direction of the Hadoop implementation.

What will make you the ideal candidate?

Historically coming from a strong unix or database administration or support background and having moved in to Big Data is what is we are looking in you. You will be experienced on Hadoop administration with supporting multi-terabyte Hadoop Infrastructure and associated Apache project. Your Hadoop skills will include but not limited to HDFS, Hive, Falcon, HBase, Oozie, Spark, Ambari, Ranger, Beeline, Kafka. You will have a strong experience in providing technical support to both technical and non-technical colleagues and customers. You will have a strong experience in Linux, including, command line editors, scripting (shell, Python, Perl, etc).

Why Worldpay?

Worldpay has merged with Vantiv Inc. to create a new global leader in payments. The new Worldpay will have the scale, the reach and the resources to take our success story to another level. This move allows us to expand further into high growth markets, deliver innovation at scale and above all, brings together exceptional talent and a shared passion that will help propel us forward to the next chapter of our collective story.


Our technology enables billions of payments to be made each year whether online or in person. Working with customers large and small we help them to take payments quickly, safely and reliably, allowing them to grow their businesses and making your life more convenient in the process.



Job Number



London / England

Business Area

Tech and Engineering

Employment Type



January 25, 2018


February 25, 2018