© 2017 Altair Engineering, Inc. Proprietary and Confidential. All rights reserved.
목차
왜 HPC 클라우드인가?
Microsoft Azure란?
Azure 사용시나리오
적용사례
추가정보
© 2017 Altair Engineering, Inc. Proprietary and Confidential. All rights reserved.
왜 HPC 클라우드인가?
© 2016 Altair Engineering, Inc. Proprietary and Confidential. All rights reserved.
전통적인과학/공학분야는물론, 4차산업혁명의신생분야에서도 HPC는핵심역량입니다.
RenderingSimulation Visualization Deep-Learning/AI
• Finite Element Analysis (FEA)
• Computational Fluid Dynamics (CFD)
• Multibody Dynamics (MBD)
• Simulation Program with Integrated Circuit Analysis (SPICE)
• Weather/Climate
• Concept Vehicle Styling
• Digital Concept Vehicle
• Replace clay models
• Marketing Campaign
• Web, media, print
• Engineering
• E.g., cockpit glare
• 3D Animation
• Cloud-based engineering (cloud VDI)
• Cloud-based post visualization of simulations
• Virtualized Reality
• Augmented Reality
• Autonomous driving
• Advanced Driver Assistance Systems (ADAS)
• Virtual Assistant
• Personalized Healthcare
• Smart CCTV
• Social Network
• Digital Forensics/Intelligence
© 2016 Altair Engineering, Inc. Proprietary and Confidential. All rights reserved.
구분 Top500* 슈퍼컴퓨터 실용적HPC 시스템
도입목적 • 거대문제풀이 (Grand
Challenge Problems)
• 생산성극대화 (Productivity)
사용자수 • 소수 • 다수
작업시간 • 수일 ~ 수개월 • 수시간 ~ 수일
코어수 • 10,000 ~ 10,000,000 • 100 ~ 10,000
연산성능 • 100 TF ~ 100 PF 이상 • 1 TF ~ 100 TF 이상
연간비용 • 10억원 ~ 100억원이상 • 1억원 ~ 10억원이상
범용성 • 낮음 • 높음
기술특성 • 한계를극복하는신기술 • 안정성이검증된기술
우선순위 • 최고성능구현 • 적시의결과생산
작업수
문제크기
실용적 HPCTop500
실용적 HPC는생산성극대화를위해적시성, 신뢰성, 안정성이요구됩니다.
© 2016 Altair Engineering, Inc. Proprietary and Confidential. All rights reserved.
Cloud MaturityBetter security and cost competitive
Leverage high end GPU’s for visualization work
without the cost of buying hardware
Azure HPC VMs are often as fast, or sometimes
faster than bare metal clusters
AgilityTime to market for customers is becoming more
critical than building datacenters
CostPay only for what you use, cores available on
demand.
tCo
mp
ute
Inactivity
Period
1. On and Off
4. Growing Fast
2. Unpredictable
Bursting
3. Predictable Bursting
6. 24x7 Steady5. Legacy Decline
더안전하고, 신속하고, 유연하고, 경제적인 HPC 환경을위해클라우드를도입하고있습니다.
© 2016 Altair Engineering, Inc. Proprietary and Confidential. All rights reserved.
역할 전략의변화 직접효과 간접효과
관리자 시장추종
-> 시장도전
• 초기/고정 투자최소화• 제품개발주기단축• 제품품질향상
• 경쟁력제고✓ 적시성✓ 차별성✓ 유연성
운영자 인프라구축
-> 서비스연결
• 설치/안정화 기간단축• 장애예방/정비불필요• 보안취약점최소화
• 서비스최적화✓ 안정성✓ 효율성✓ 보안성
사용자 자원중심업무관리
-> 업무중심 자원관리
• 대기시간없는작업환경• 무제한의각종자원접근• 작업완료시간예측용이
• 생산성향상✓ 신뢰성✓ 응답성✓ 다양성
HPC클라우드도입의효과는사용자, 운영자, 관리자에걸친모든조직에변화를일으킵니다.
© 2017 Altair Engineering, Inc. Proprietary and Confidential. All rights reserved.
Microsoft Azure란?
© 2016 Altair Engineering, Inc. Proprietary and Confidential. All rights reserved.
42Azure regionsMore than any other cloud provider
US Gov Texas and US Gov Arizona
NEWLY ANNOUNCED: France: France Central and France South
Africa: South Africa North and South Africa West
Australia: Australia Central 1 and Australia Central 2
전세계 Microsoft Azure 리전
Korea South
Korea Central
Korea
© 2016 Altair Engineering, Inc. Proprietary and Confidential. All rights reserved.
Busan
Seoul
ExpressRoute - Azure의고속접속네트웍
Korea
© 2016 Altair Engineering, Inc. Proprietary and Confidential. All rights reserved.
© 2016 Altair Engineering, Inc. Proprietary and Confidential. All rights reserved.
US G
ov
HIPAA /
HITECH
Act
Moderate
JAB P-ATO FIPS 140-2
FERPA
DoD DISA
SRG Level 2 ITAR CJIS
GxP
21 CFR Part 11
IRS 1075Section
508
VPAT
Glo
bal
ISO 27001
SOC 1
Type 2ISO 27018CSA STAR
Self-Assessment
Reg
ion
al
Singapore
MTCS
UK
G-CloudAustralia
IRAP/CCSL
FISC
Japan
China
DJCP
New
Zealand
GCIO
China
GB
18030
EU
Model Clauses
ENISA
IAFArgentina
PDPA
Japan CS
Mark
Gold
SP 800-171
China
TRUCS
Spain
ENS
Ind
ust
ry
PCI DSS
Level 1 CDSAShared
AssessmentsMPAA
Japan My
Number Act
FACT
UK
High
JAB P-ATO
GLBA
DoD DISA
SRG Level 4
MARS-E FFIEC
ISO 27017SOC 2
Type 2SOC 3
India
MeitY
Canada
Privacy
LawsPrivacy
Shield
ISO 22301
Germany IT
Grundschutz
workbook
Spain
DPA
CSA STAR
Certification
CSA STAR
Attestation
HITRUSTIG Toolkit
UK
DoD DISA
SRG Level 5
Azure는규정을얼마나준수하는가?
Azure는 54가지의세계적인규정을준수하여업계최고의신뢰성을보장합니다.
© 2016 Altair Engineering, Inc. Proprietary and Confidential. All rights reserved.
Azure는공개소프트웨어와리눅스를적극적으로지원하여선택의폭을극대화했습니다.
© 2016 Altair Engineering, Inc. Proprietary and Confidential. All rights reserved.
Azure에서는각종해석솔버, 가시화, 스케줄링, 개발도구, 파일시스템솔루션이지원됩니다.
• Preferred Scheduler
✓ PBS-Pro/Ccompute Manager
✓ LSF/Symphony
✓ Microsoft HPC Pack
• Supported MPI
✓ Intel MPI
✓ MS-MPI on Windows
• Parallel file systems
✓ Intel Lustre
✓ BeeGFS
✓ GlusterFS
Workload Major Applications
Crash/Dynamics/ImpactRADIOSS® , LS-DYNA® , PAM-
CRASH® , Abaqus/Explicit®
CFD
AcuSolve® , Abaqus® , Ansys
Fluent® , STAR-CCM+® ,
OpenFOAM® , ANSYS CFX®
Structural
Abaqus/Standard® ,
OptiStruct® , ANSYS
Mechanical® , MSC
NASTRAN®
VisualizationParaview® , Blender® , CEI® ,
Fieldview®
• Operating System
✓ Linux
✓ Windows
© 2016 Altair Engineering, Inc. Proprietary and Confidential. All rights reserved.
Entry Level VMs
Dev/Test Workolads
A
General Purpose VMs
Common Applications, Web servers etc
D
Compute Optimized VMs
Gaming, Analytics
F
Large Memory VMs
Large Databases
G
>80,000 IOPsPremium Storage
Low latency, high throughput apps
High Performance VMs
Batch processing, fluid dynamics, monte carlo simulation
H
GPU-enabled VMs
NV -Graphic based applications
NC2* – Advanced Sim (P100-X)
ND1* – AI Inferencing (P40)
ND2* – AI Training (V100)
N
Storage optimized VMs
No SQL Databases (Cassandra, MongoDB), Data warehousing
L
*Announced – Private Preview from CY17 H2
Azure는사용자의작업특성별요구사항을만족하기위해다양한 VM을지원하고있습니다.
© 2016 Altair Engineering, Inc. Proprietary and Confidential. All rights reserved.
H
• A8-A9 and A10-A11 are
performance-tuned VMs that
provide bare-metal results
• 8 or 16 CPU cores per VM
• A8-A9 with QDR InfiniBand +
RDMA on Windows and Linux
• < 3 µsecs latency
• 32 Gbps throughput (actual)
• Fully non-blocking topology to
ensure max bandwidth
• E5-2670 Sandy Bridge processor
A N
• H series
• Fastest VMs in the public cloud
• 8 or 16 CPU cores per VM
• H16r and H16mr with FDR
InfiniBand + RDMA on Windows
and Linux
• <2.5- 3.5 µsecs latency
• Fully non-blocking topology
• E5-2667 v3 Haswell processor
• Up to 224 GB DDR 4 memory
• Up to 2 TB of local SSD
• NC and NV series
• Up to 4 GPUs per VM
• Up to 24 CPU cores per VM
• NC24r with FDR InfiniBand +
RDMA on Windows and Linux
• < 2.5-3 µsecs latency
• Fully non-blocking topology
• E5-2667 v3 Haswell processor
• Up to 224 GB DDR 4 memory
• Up to 2 TB of local SSD
Azure는 HPC를위해특별히고성능 CPU, GPU, Infiniband를탑재한다양한 VM을제공합니다.
© 2016 Altair Engineering, Inc. Proprietary and Confidential. All rights reserved.
https://www.hpcwire.com/2017/02/15/hpc-clouds-ready-azure-edges-aws-benchmark-study/
HPC시장분석전문지에따르면 Azure에서측정한 Linpack벤치마크성능이가장높았습니다.
© 2016 Altair Engineering, Inc. Proprietary and Confidential. All rights reserved.
0
2000
4000
6000
8000
10000
12000
16 32 64 96 128 160 192 224
RU
N T
IME
IN
SE
CO
ND
S
NUMBER OF CORES
Radioss Crash Simulation code results(Lower is better)
Linux RDMA On Azure Bare metal
Azure의가상화는대부분의실제애플리케이션수행성능에영향을거의미치지않습니다.
© 2016 Altair Engineering, Inc. Proprietary and Confidential. All rights reserved.
Azure는 Infiniband를제공하여코어를많이사용해도선형적으로성능이증가합니다.
© 2016 Altair Engineering, Inc. Proprietary and Confidential. All rights reserved.
Azure는 Infiniband를제공하여코어를많이사용해도선형적으로성능이증가합니다.
© 2017 Altair Engineering, Inc. Proprietary and Confidential. All rights reserved.
Azure 사용시나리오
© 2016 Altair Engineering, Inc. Proprietary and Confidential. All rights reserved.
Cloud burst : IaaS/PaaS
• Burst to cloud to add resources when
needed
• Continue using your existing on-premises
infrastructure
• Run workloads on Windows and Linux, on
Azure and on-premises
• Cover peaks in demand or special projects
• Pay only for what you use
• Microsoft HPC Pack, Univa Grid Engine,
and Altair PBS Pro already have this
capability (and more are coming!)
Compute nodes in the cloudOn-premises
Client
Add cloud resources to
your cluster, on demand
온프레미스 HPC시스템의성능이부족할때,클라우드로작업을분산할수있습니다.
© 2016 Altair Engineering, Inc. Proprietary and Confidential. All rights reserved.
Cloud clusters : IaaS
All HPC resources in the cloudOn-premises
Client
• Deploy a complete HPC cluster, all in the
cloud, in just minutes
• Use templates, scripts, and images to
quickly deploy at any scale
• Use your current HPC scheduler
• Shift existing applications, scripts and
tools to cloud
• Deploy as many clusters as you need!
Provision one (or more)
new clusters in minutes
Azure에온프레미스와같은 HPC 클러스터환경을구축할수있습니다.
© 2016 Altair Engineering, Inc. Proprietary and Confidential. All rights reserved.
Cloud Workstations : IaaS
Remote workstations on the cloudOn-premises
• Deploy one or more GPU-enabled
workstations on the cloud
• Connect from any laptop or device (iOS,
Android, Windows)
• Collaborate, share, explore.
• Run windows server 2016 or Linux OS
• With or without optimized remote
visualization platforms (e.g., Teradici
PCoIP or Citrix HDX)
Deploy powerful GPU-
enabled workstations
NV NVNV
NV NV
Tablet
Zero
client
Azure의 GPU VM을사용하여원격그래픽워크스테이션환경을구축할수있습니다.
© 2016 Altair Engineering, Inc. Proprietary and Confidential. All rights reserved.
Azure Marketplace: 주문형 HPC 솔루션
Data services
Virtualmachines
App services
Web appsAAD apps
Azure marketplace• Simple deployment for pre-packaged
solution
• A unified location for Azure based offerings
from Microsoft and partners
• Thousands of offers
• Integrated platform experience Streamlined
configuration, deployment, and
management
• Fortune 500 and SMB customers across 86
global markets
The Azure marketplace brings Azure partner ecosystem to customers around the world
Azure Marketplace에서이미검증된 HPC 솔루션을선택하여즉시사용할수있습니다.
(https://azuremarketplace.microsoft.com/en-us/marketplace/)
© 2016 Altair Engineering, Inc. Proprietary and Confidential. All rights reserved.
HPC as a Service: SaaS
All HPC resources in the cloudOn-premises
• Integrate with Azure Batch, directly from a
client application (GUI or CLI) or online
portal
• Batch abstracts resource management and
scheduling completely
• Supports small to extremely large
deployments and can deploy any VM size
• Provides auto-scaling and stopping of
resources
• Run HPC jobs at scale on Docker
containers
• Using Batch is free, you only pay for the
underlying compute!
어떤종류의 VM이든, 어떤규모의작업이든웹포털이나클라이언트애플리케이션을통해직접요청합니다.
Client
Client App or Web portal
AzureBatch
클라우드환경을직접구성할필요없이클라이언트에서작업을제출하면 Azure가수행합니다.
© 2016 Altair Engineering, Inc. Proprietary and Confidential. All rights reserved.
Preview
NCTesla K80
A/D/F SERIES
NC_v2
Tesla P100
Embarrassingly Parallel Rendering Tasks
• What is it?• Easy to run ray traced rendering workloads utilizing
commercial renderers such as V-Ray and Arnold via applications such as Autodesk Maya and 3ds Max.
• Run jobs on either GPU or CPU based VMs
• Built on Azure Batch & Azure Compliant platform
• Designed & built with ISV partners such as Autodesk, NVIDIA and Chaos Group.
•
• Key Capabilities• Support for V-Ray & Arnold on GPU or CPU rendering on
both Linux and Windows including pay-per-use licensing (per min/hour) for all commercial applications.
• Easy to use client plugins in Autodesk Maya & Autodesk 3ds Max on Linux, Windows or Mac OS.
• Auto-upload to storage, monitoring of jobs, management of pools, job configuration and auto-output download features built into extensible Plugin UI for applications.
• Pools can run Low-Priority and Normal VMs with renderers.
• Rendering Python API to integrate into automated pipelines
• Azure Portal experience for high-level user and admin scenarios
• H1 CY 2017• Already in Private Preview, with GA late Summer
• Testing with enterprise external customers in Preview today.
Integrated Application Plugin Experience- Job Submission- Job Monitoring- Asset Management & Sync- Pool Management- Environment Configuration
Azure Batch
Upload assets
Submit job
Return outputs
Azure클라우드에서 Rendering 소프트웨어와그래픽처리를위한 VM을결합하여실시간으로서비스를사용할수있습니다. (https://rendering.azure.com/)
© 2016 Altair Engineering, Inc. Proprietary and Confidential. All rights reserved.
NcFP32/64 TRAINING
A/D SERIES
AZURE BATCH
ND2INT32/64 TRAINING
ND1INT32
TRAINING
INFINIBAND HIGH SPEED NETWORK
FPGA1ACCELERATED
APPS
FPGAMICROSERVICES
HAASMGMT
Rendering Service
TENSOR FLOW
CNTK MXNET CAFEE
• What is it?
• Easy to run parallel large-scale training with GPU and InfiniBand high speed networking
• Experimentation and simulation with any framework
• Built on Azure Batch, Docker & Python
• Azure Compliant Infrastructure
• Designed with and supported by AIR Philly Team
•
• Key Capabilities
• Open- TensorFlow, CNTK, Caffe, MXNet support
• Submit and go parameterized job configuration
• Experiment management and monitoring
• Support teams with sharing of reserved capacity
• Re-image VMs between users for compliance
• Python API to integrate with experimentation tools
• Publish models for eval with AML
• H1 CY 2017
• Preview mid semester, summer GA –In Preview
• Good support for internal and external early adopters
AZURE BATCH
DEEP LEARNING JOB SERVICE
TENSOR FLOW
CNTK MXNET CAFEE
개발중
대규모 AI Training 작업을 Azure Batch를사용하여쉽게구현해줍니다. (https://batchaitraining.azure.com/)
© 2016 Altair Engineering, Inc. Proprietary and Confidential. All rights reserved.
애플리케이션수행단계 : 수동으로할경우
Get and manage VMs
Start the tasks
Move task input and output Queue tasks
Install task applications
Scale up and downTask failure? Task frozen?
Manage and authenticate users
Significant amount effort
spent managing compute
resources, security, data
movement, job running,
and application lifecycle,
not related to your actual
workload or business
User application or service
PaaS
Cloud Services
IaaS
Virtual Machines
Hardware
Provided by the cloud
platform
PaaS나 IaaS로애플리케이션을수행하기위해서는복잡한준비단계를직접처리해야합니다.
© 2016 Altair Engineering, Inc. Proprietary and Confidential. All rights reserved.
Azure Batch: HPC as a Service
User application or service
PaaS
Cloud Services
IaaS
Virtual Machines
Hardware
Azure Batch
VM management and job scheduling
App lifecycle, job dependencies, data movement,
task rescheduling, user management & authorization
• Don’t worry about the “plumbing”
• Focus on the workload/app
• Access higher-level capabilities
• Minimize the required cloud or
Azure experience
Provided by the cloud
platform
Azure Batch를사용하면사용자에의한중간단계의준비과정없이자동으로, 즉시 VM을생성하고애플리케이션을수행할수있습니다. (https://docs.microsoft.com/en-us/azure/batch/)
© 2016 Altair Engineering, Inc. Proprietary and Confidential. All rights reserved.
Azure Batch를이용한 SaaS 구현
Client plugins
Application & files
Job splitter & task processor
Batch SDK를사용하여클라우드를구현하고애플리케이션업로드
개발자
운영자• 사용자/그룹관리
• 역할기반접근
• 작업및 VM 모니터링
• 사용량보고서
• 쿼터관리
• 문제해결
사용자 • 클라이언트에서작업제출
• 입력/출력데이터동기화
• 플러그인/포털을통해작업모니터링
AZURE BATCH
AZURE STORAGE
Azure Batch의자동클라우드생성및애플리케이션수행기능은개발자, 사용자, 운영자모두에게편리합니다.
© 2017 Altair Engineering, Inc. Proprietary and Confidential. All rights reserved.
적용사례
© 2016 Altair Engineering, Inc. Proprietary and Confidential. All rights reserved.
Seeing AIHendricksMotor Sports
© 2016 Altair Engineering, Inc. Proprietary and Confidential. All rights reserved.
Azure GPU 기반으로AutoCAD를태블릿 PC에서사용
•가장많은성능을 요구하는 애플리케이션들을 클라우드에서 수행함으로써고성능 워크스테이션 환경구축
•클라우드 기반아키텍처를 이용하여 디자이너, 엔지니어, 컨텐츠 제작전문가들이 어떤 장비의웹브라우저에서든 3D 애플리케이션 수행
• Microsoft Azure의 NVIDIA Tesla GPU 기반 N-시리즈 VM 사용
•업계최고성능의 워크스테이션 애플리케이션 서비스제공
FRAME
© 2016 Altair Engineering, Inc. Proprietary and Confidential. All rights reserved.
Altair
© 2016 Altair Engineering, Inc. Proprietary and Confidential. All rights reserved.
추가정보
•Microsoft Azure Portal
https://ms.portal.azure.com
•Microsoft Azure 홈페이지https://azure.microsoft.com/ko-kr/
•Microsoft Azure 무료계정만들기https://azure.microsoft.com/ko-kr/free/
•Microsoft Azure HPC 홈페이지https://azure.microsoft.com/ko-kr/solutions/big-compute/
© 2016 Altair Engineering, Inc. Proprietary and Confidential. All rights reserved.
Top Related