BDE SC4 Hangout - Hajira Jabeen, general architecture
-
Upload
bigdataeurope -
Category
Technology
-
view
396 -
download
0
Transcript of BDE SC4 Hangout - Hajira Jabeen, general architecture
BDE Platform ArchitectureSC 4 Webinar27 June 2016
Agenda
◎ Platform architecture
◎User interfaces
Platform architecture
Platform architecture
DEPRECATED
Platform architecture
Platform architecture
Platform installation
◎ Manual installation guide
◎Using Docker Machineo On local machine (VirtualBox)o In the cloud (AWS, DigitalOcean, Azure)o Bare metal
Platform componentsBase Docker images:
◎Serves as a template for a (Big Data) technology
◎Easily extendable with custom algorithm/data
◎Automating as much as possible
◎May be opinionated
Platform components◎Published components
o Image repositories on GitHubo Automated builds on Docker Hubo Documentation on BDE Wikio docker-compose snippet in README
Developing a Pipeline
Big Data pipelines◎Run pipeline: $ docker-compose up
◎Orchestrator required for initialization processo Components may depend on each othero Components may require manual
intervention
Platform architecture
User Interfaces
User interfaces◎Workflow UIs
o Workflow Builder❖Build a workflow to configure the init daemon
serviceo Workflow Monitor
❖Monitor a pipeline's initialization process
◎Swarm UIo Deploy a pipeline on the platform
◎ Integration of existing UIso BDE CSS based on Materialize CSSo Integrator UI: present GUIs in a unified
interface
◎UI to follow-up status of the pipelineo Status of each componento Manually finish a step in the pipeline
◎ Integrated as component inthe application's docker-compose.yml
Pipeline monitor application
Thank you