Big Data Designer

Summary about this job

Detail information about job Big Data Designer. Terms and conditions vacancy

Key Responsibilities:

• Provide design input into further developing the capabilities of BT’s principal advanced security platforms and offerings.
• To provide analytical and innovative input to projects.
• Upskill team members in the domain of Big Data platforms and development.
• Provide input and thought leadership into big data platform architecture and design.
• Provide input into data management design.
• Represent BT at customer workshops.
• Maintain knowledge of the Hadoop ecosystem and related emerging technologies.
• Maintain knowledge of current trends in data handling techniques including life cycle and security maintenance.
• Maintain knowledge of security and cyber telemetry, log feeds and intelligence platforms.
• Maintain knowledge and awareness of current threat trends and security technologies.

Skills/Experience:

• Detailed understanding of the various open source projects that constitute a comprehensive big data platform.
• Extensive experience with supporting large production Hadoop platforms.
• Great understanding on how Spark works and demonstrated experience using it in production environments.
• Experience of the Elastic stack, including Elasticsearch, Logstash and Kibana.
• Experience designing, creating, optimising and troubleshooting batch ETL processes.
• Experience building and supporting high velocity streaming ingestion frameworks.
• Experience with SQL-based datastores.
• Experience with NoSQL-based datastores a bonus.
• The ability to architect and design both streaming and batch use cases in a big data platform.
• Knowledge of security and cyber telemetry, log feeds and intelligence platforms.
• Experience with deploying and managing Hadoop environments within AWS is desirable.