141204 upload
-
Upload
quey-liang-kao -
Category
Software
-
view
177 -
download
1
Transcript of 141204 upload
About Me
• Quey-Liang Kao
• Ph.D. Student
– SCOPE Lab, NTHU
– Advisor: Che-Rung Lee
• Research Interests
– Parallel Computing
– GPGPU
About This Project
• Experimental project with Dr. Chung and Dr. Seelam in IBM Research
• Testing & benchmarking on HPC clusters
• Since this July
• A poster at “Emerging Tech” session of SC14
• Still ongoing
Outline
• Brief History of Docker
• Basic Docker Tutorial / Use Case
• HPC on Docker
• SC14 @ New Orleans, LA
Before March 2013
• A private project in dotCloud
– A PaaS company
– For internal use as an infrastructure
Aug 2014
• Docker, Inc. sold dotCloud to cloudControl [2]
– Another PaaS company
– “We want to concentrate on Docker”
Why?
• As promising as java and git [1]– “Write Once. Run Anywhere.”
– “Tiny footprint with lightning fast performance”
• Speed up deployment
• Eco system– cloudControl, PaaS companies
– Red Hat, Microsoft, …
– Amazon
– etc.
What is Docker exactly? [12]
• Docker is a powerful tool to do with containers and images
• What is container?– Linux Container
– kernel feature: cgroup, namespace
– OS-level virtualization
• What is image?– Which provides the content of file systems
for containers
Install
• From source: use docker to build docker
• From pre-built package [9]
– Depends on your linux distribution
– Mac, Windows are supported
Run
• Only one command: docker
– Both for daemon and CLI
• Daemon
– service docker start, or
– systemctl start docker
– common argument: -b=bridge0
• CLI
Manipulate Images
• docker images
– view images
• docker pull/push <image tag>
– pull/push images from docker hub
• docker search <keyword>
– search images matching keyword
Manipulate Images
• docker rmi <image tag>
– remove the image
– if there are containers using this image, then this may not work
• docker rmi $(docker images -q --filter "dangling=true")
– useful combo to eliminate zombie images
Images? Containers?
Active(An Instance)
Static(A Model)
Operating System
Process Program
DockerInfrastructure
Container Image
Manipulate Containers
• docker ps [|-a]– check [alive|all] containers
• docker run -i -t <image> <exec>– launch an instance of the image and run the
executable
– use ctrl-p ctrl-q to detach
– add -d argument to run in detach mode
• docker attach/detach <contID>– as the command
Manipulate Containers
• docker top
– check running process
• docker start/stop <ContID>
– as the command
• docker rm <ContID>
– remove the container
Manipulate Containers:git-style experience
• docker commit <contID> <Image>
– commit changed container to a new image
• docker diff <contID>
– comapre the changes of the container to original image
Manipulate Containers:java-style experience
• Dockerfile
– A text file that describes how to build an image
– There have been many public Dockerfiles
• docker build -t <image> <path/to/Dockerfile>
– build an image according to Dockerfile
– portable
HPC on Cloud
• Cloud– Isolation
– Resource control
– IaaS / Virtualization
• Containers– OS-level Virtualization
– Light-weighted
– namespace
– cgroup
Motivation
• Examine the performance of container-based cluster
• Experience the cost of deploying containers (rather than VMs)
• Catch up with the trend
Environment Setup
• Native
– Four-node cluster, with 24 CPUs each
• VM
– Use qemu as the hardware simulator, and accelerate with KVM kernel module
• Container
– Use docker
Benchmarks
• HPL(High Performance Linpack)
– LU decomp. For large matrix
– Benchmark for Top 500 SuperComputers
– Measured in G/T/PFLOPs
Benchmarks
• NPB(NAS Parallel Benchmark)
– Integer sort
– Embarrassingly parallel
– 3D Fourier transform
– etc.
• Network, I/O benchmarks
– not presented here
Conclusion
• Containers performs well for computation-intensive apps
– since they are processes with namespaces
• We reproduce some of the experiment results of an report from IBM [11]
Future Work
• IBM side
– Try their POWER-based supercomputers/clusters
• My side
– GPU support? (Surveying)
– Migration? (Surveying)
– ?
References
1. http://www.centurylinklabs.com/what-is-docker-and-when-to-use-it/
2. http://techcrunch.com/2014/08/04/docker-sells-dotcloud-to-cloudcontrol-to-focus-on-core-container-business/
3. Many Wikipedia pages, e.g. Docker, linuxcontainers, cgroups, namespace, etc.
4. http://www.slideshare.net/dotCloud/docker-resentation-meetupparis
References
5. http://azure.microsoft.com/blog/2014/10/15/new-windows-server-containers-and-azure-support-for-docker/
6. http://aws.amazon.com/cn/about-aws/whats-new/2014/04/23/aws-elastic-beanstalk-adds-docker-support/
7. All documents here: http://docs.docker.com/
8. http://blog.docker.com/2014/11/announcing-docker-global-hack-day-2-winners/
References
9. https://docs.docker.com/installation/
10.http://www.nas.nasa.gov/publications/npb.html
11.An Updated Performance Comparison of Virtual Machines and Linux Containers
12.http://www.slideshare.net/adrianjotto/docker-101-38986794?qid=52aa9f3a-2dd1-4b1f-811d-ff0b0f8cc20b&v=qf1&b=&from_search=6