IPSAP/ EXPLICIT VIBRATION ANALYSIS STRESS ANALYSIS 기계항공 공학부, 서울 대학교 김 승...

52
IPSAP/ EXPLICIT VIBRATION ANALYSIS STRESS ANALYSIS 기기기기 기기기 , 기기 기기기 기 기 기 2006, 11, 21 Supercomputing Korea 2006 On Computational Structures Technologies Aero STructures Lab
  • date post

    19-Dec-2015
  • Category

    Documents

  • view

    218
  • download

    0

Transcript of IPSAP/ EXPLICIT VIBRATION ANALYSIS STRESS ANALYSIS 기계항공 공학부, 서울 대학교 김 승...

Page 1: IPSAP/ EXPLICIT VIBRATION ANALYSIS STRESS ANALYSIS 기계항공 공학부, 서울 대학교 김 승 조 2006, 11, 21 Supercomputing Korea 2006 On Computational Structures Technologies.

IPSAP/EXPLICIT

VIBRATION ANALYSIS

STRESS ANALYSIS

기계항공 공학부 , 서울 대학교김 승 조

2006, 11, 21

Supercomputing Korea 2006

On Computational Structures Technologies

Aero STructures Lab.

Page 2: IPSAP/ EXPLICIT VIBRATION ANALYSIS STRESS ANALYSIS 기계항공 공학부, 서울 대학교 김 승 조 2006, 11, 21 Supercomputing Korea 2006 On Computational Structures Technologies.

http://aeroguy.snu.ac.kr2/52

Introduction

History of Finite Element Analysis

Representative Structural Analysis Codes

– Large Scale Parallel Structural Analysis Code

– Commercial FEM Packages

High-Performance FE Software, IPSAP

Research Trend of Virtual Design and Development

Conclusion

Contents

Page 3: IPSAP/ EXPLICIT VIBRATION ANALYSIS STRESS ANALYSIS 기계항공 공학부, 서울 대학교 김 승 조 2006, 11, 21 Supercomputing Korea 2006 On Computational Structures Technologies.

http://aeroguy.snu.ac.kr3/52

Computers have been widely used in structural engineering for:

– Structural analysis– Computer-aided design and drafting (CADD)– Report preparation

Typical computer usage by an engineer:– Word-processing– Preparation of tender documents and engineering drawings– Small and intermediate computations– Analysis of structures– Design work– Data reduction and storage– Software development– Email– Etc.

Introduction

Page 4: IPSAP/ EXPLICIT VIBRATION ANALYSIS STRESS ANALYSIS 기계항공 공학부, 서울 대학교 김 승 조 2006, 11, 21 Supercomputing Korea 2006 On Computational Structures Technologies.

http://aeroguy.snu.ac.kr4/52

Introduction of Structural Analysis

Computational Structural Analysis : The use of computerized methods to predict the response,

performance, failure and service life of structures and materials under various types of loading conditions

The role of Computational Structural Analysis– Allow the simulation of the behavior complex systems beyond

the reach of analytic theory.– Provide detailed design information in a timely fashion.– Enhance our understanding of engineering systems by

expanding our ability to predict their behavior.– Provide the ability to perform multidisciplinary design

optimization.– Increase competitiveness and lower design/production costs.

Page 5: IPSAP/ EXPLICIT VIBRATION ANALYSIS STRESS ANALYSIS 기계항공 공학부, 서울 대학교 김 승 조 2006, 11, 21 Supercomputing Korea 2006 On Computational Structures Technologies.

http://aeroguy.snu.ac.kr5/52

Introduction of Structural Analysis

구조해석은 흔히들 생각하는 다리와 건물의 구조를 해석하는

토목공학분야뿐만 아니라 항공우주공학 , 기계공학 , 선박해양공학 ,

전자공학 등의 다양한 분야에서 기계적 요소들로 이루어진 구조물의

해석까지 포함

구조해석에서 유한요소 해석법 (FEM:Finite Element Method) 이 가장

보편적으로 사용되는 방법 .

Page 6: IPSAP/ EXPLICIT VIBRATION ANALYSIS STRESS ANALYSIS 기계항공 공학부, 서울 대학교 김 승 조 2006, 11, 21 Supercomputing Korea 2006 On Computational Structures Technologies.

http://aeroguy.snu.ac.kr6/52

Korea market share of CAE in 2004 (45 million $)

CAD & Graphics

OS % of automotives

SC2005 (Top500), ‘Top20Auto Survey of HPC Installations in

the Automotive’

Introduction of Structural Analysis

Current Market for CAE Software– Growth of CAE on industrial field– Especially, structural analysis has largest percentage(50%) of it.– This grow up over time going. In Korea, CAE market in 2005 grew

16% larger than that in 2004.

the expectation ofstructural analysis software market

Page 7: IPSAP/ EXPLICIT VIBRATION ANALYSIS STRESS ANALYSIS 기계항공 공학부, 서울 대학교 김 승 조 2006, 11, 21 Supercomputing Korea 2006 On Computational Structures Technologies.

http://aeroguy.snu.ac.kr7/52

Matrix method : Martin(1966), Meek(1971), Wilson(1960) Terminology of Finite Element : Clough(1960) Extended to all kinds of problems described as variational formulation : Zien

kiewicz Development of efficient finite elements, nonlinear and dynamic analysis

: Oden, Bathe, Huges, Zienkiewicz, Belytschko, Crisfield et al Commercial FEA packages : NASTRAN(1963), ABAQUS(1978), ANSYS(1970) Related Industry :

Manufacturing/Machinery, Automotives, Rail/Transportation, Aerospace/Defense, Consumer/Electronics Products, Medical/Biomechanics, Rubber/Sealing

History of Finite Element Analysis

Introduction of Structural Analysis

Page 8: IPSAP/ EXPLICIT VIBRATION ANALYSIS STRESS ANALYSIS 기계항공 공학부, 서울 대학교 김 승 조 2006, 11, 21 Supercomputing Korea 2006 On Computational Structures Technologies.

http://aeroguy.snu.ac.kr8/52

Iterative method for sparse system– Iterative computations of matrix-vector operations– Jacobi algorithm, Gauss-Seidel algorithm, Conjugate Gradient Method, D

DM(FETI), etc.– Converging speed affected by matrix condition

Operation count can not be estimated beforehand– Easy to parallelize efficiently

Direct method for sparse system– Based on Gauss elimination (LU factorization)– Operation counts determined by matrix size and non-zero pattern (mesh c

onnectivity) Operation count can be estimated beforehand

– Band, skyline, frontal, multifrontal solver, etc.– Numerical robustness, Multiple RHS– Difficult to parallelize efficiently

Introduction of Structural Analysis

Page 9: IPSAP/ EXPLICIT VIBRATION ANALYSIS STRESS ANALYSIS 기계항공 공학부, 서울 대학교 김 승 조 2006, 11, 21 Supercomputing Korea 2006 On Computational Structures Technologies.

http://aeroguy.snu.ac.kr9/52

세계적 최신 전산 해석 프로그램의 개발 및 적용 사례

EU• Dassault 사 Falcon 7x• AirBus 사 가상구조시험

일본•Adventure 프로젝트 •GeoFEM

미국• 각종 유명 상용코드 독점• SALINAS 프로젝트

Introduction of Structural Analysis

Page 10: IPSAP/ EXPLICIT VIBRATION ANALYSIS STRESS ANALYSIS 기계항공 공학부, 서울 대학교 김 승 조 2006, 11, 21 Supercomputing Korea 2006 On Computational Structures Technologies.

http://aeroguy.snu.ac.kr10/52

Large Scale Parallel Structural Analysis Code

SALINAS project - USA– 미국 에너지성의 ASCI 프로젝트의 일환– Developed by Sandia National Laboratory in 1999– 1~10 억 자유도를 필요로 하는 매우 복잡한 구조물의 응력 , 진동 , 과도응답 유한요소

모델에 대한 확장성이 강한 계산 도구의 제공을 목적– Implicit Solver 를 기반으로 하여 수천개 이상의 프로세서로 구성된 ASCI 시스템에서 활용– DDM 에 기반을 둔 Multigrid 알고리듬과 coarse auxiliary 알고리듬과 같은 Multilievel iterati

ve scheme 을 참고한 FETI-DP 알고리듬을 이용– Scalability 를 중시하여 1,000 여개 계산 노드를 가지는 ASCI Red, white 등의 시스템에서

순환적으로 운용

Page 11: IPSAP/ EXPLICIT VIBRATION ANALYSIS STRESS ANALYSIS 기계항공 공학부, 서울 대학교 김 승 조 2006, 11, 21 Supercomputing Korea 2006 On Computational Structures Technologies.

http://aeroguy.snu.ac.kr11/52

Large Scale Parallel Structural Analysis Code

ADVENTURE Project - JAPAN– ADVanced ENgineering analysis Tool for Ultra large REa

l world– Development of Computational Mechanics System for L

arge Scale Analysis and Design– 1997 ~ 2002– 목표 : 슈퍼 컴퓨터 (MPP, PC Cluster) 로 천만 ~ 일억

자유도를 가지는 임의의 모델 형상을 1 시간 ~ 하루 정도의 시간으로 해석 가능

– 약 20 여개의 pre-processing, post-processing 모듈을 구성

Pantheon Model(1.5M DOF) Solid Analysis Fluid Analysis

Page 12: IPSAP/ EXPLICIT VIBRATION ANALYSIS STRESS ANALYSIS 기계항공 공학부, 서울 대학교 김 승 조 2006, 11, 21 Supercomputing Korea 2006 On Computational Structures Technologies.

http://aeroguy.snu.ac.kr12/52

Large Scale Parallel Structural Analysis Code

GeoFEM - JAPAN– Parallel FE Solid Earth Simulator– 1997 ~ 2003– Localized operation & optimum data structures for mass

ively parallel computation– Pluggable design– Platform : linear solver, I/O, visualization

GeoFEM Platform

A test dataset on the ES with 5,886,640 unstructured elements

Geodynamo process and fluid dynamics in the Earth’s outer core

Modeling of Philippine Sea plate boundary

Page 13: IPSAP/ EXPLICIT VIBRATION ANALYSIS STRESS ANALYSIS 기계항공 공학부, 서울 대학교 김 승 조 2006, 11, 21 Supercomputing Korea 2006 On Computational Structures Technologies.

http://aeroguy.snu.ac.kr13/52

Commercial FEA Packages

MSC NASTRAN– Developed by NASA as analysis tools for

the structural analysis of spacecraft. (1963) and managed by MSC

– Through 40 years of R&D, MSC/NASTRAN has been regarded as a standard analysis system in most area of industry.

– Capable of linear static analysis, buckling analysis, vibration and thermal analysis.

– Sparse matrix solver, Automated Component Modal Synthesis

– Analysis results of aerospace structural parts are used as the certification of quality.

Certificated by FAA (USA)

< Structural analysis of VAN >

Turbine blade thermal stress analysis

Stress analysis of Car Bumper

FE model vibration analysis

stress analysis nonlinear analysis

Page 14: IPSAP/ EXPLICIT VIBRATION ANALYSIS STRESS ANALYSIS 기계항공 공학부, 서울 대학교 김 승 조 2006, 11, 21 Supercomputing Korea 2006 On Computational Structures Technologies.

http://aeroguy.snu.ac.kr14/52

Commercial FEA Packages

MSC NASTRAN - Parallel Performance

출처 : www.mscsoftware.com , Ver. : 2004

Page 15: IPSAP/ EXPLICIT VIBRATION ANALYSIS STRESS ANALYSIS 기계항공 공학부, 서울 대학교 김 승 조 2006, 11, 21 Supercomputing Korea 2006 On Computational Structures Technologies.

http://aeroguy.snu.ac.kr15/52

Commercial FEA Packages

ABAQUS– Developed by Hibbitt, Karlsson & Sorrensen in 19

78– In 2005, Dassault Systems(CATIA) acquired ABAQ

US : SIMULIA– Linear and nonlinear structural analysis– Multifrontal solver, Block Lanczos eigen solver– Vectorized Explicit Time Integration for the dynami

c analysis– Conduction, convection and heat transfer proble

m– Analysis of offshore structure

wave-induced inertial force, buoyant force and drag of fluid

Stress analysis of airplane engine

Page 16: IPSAP/ EXPLICIT VIBRATION ANALYSIS STRESS ANALYSIS 기계항공 공학부, 서울 대학교 김 승 조 2006, 11, 21 Supercomputing Korea 2006 On Computational Structures Technologies.

http://aeroguy.snu.ac.kr16/52

Commercial FEA Packages

ABAQUS - Parallel Performance

출처 : www.abaqus.com , Abaqus ver. 6.6

E1: Car crash(274,632 elements)

E2: Cell phone drop (45,785 elements)

E3: Sheet forming (34,540 elements)

E4: Projectile penetration (237,100 elements)

단위 :sec

Page 17: IPSAP/ EXPLICIT VIBRATION ANALYSIS STRESS ANALYSIS 기계항공 공학부, 서울 대학교 김 승 조 2006, 11, 21 Supercomputing Korea 2006 On Computational Structures Technologies.

http://aeroguy.snu.ac.kr17/52

Commercial FEA Packages

– Developed by John Swanson in 1970– Utilized in conceptual design of the

product and the manufacturing process– Provides general graphic utilities– Various analysis utilities

Basic structural analysis, CFD, Electro-magnetic analysis

Thermal stress, Acoustic analysis, Piezoelectric analysis

Multi-physics– AI*NASTRAN solver

Wavefrontal solver based on sparse matrix solver,

Substructuring analysis option for large structures

– Block Lanczos eigen solver– Distributed Pre-conditioned Conjugate

Gradient (DPCG) Distributed Jacobi Conjugate Gradient (DJCG)

Engine analysis

Infrared camera analysis

Page 18: IPSAP/ EXPLICIT VIBRATION ANALYSIS STRESS ANALYSIS 기계항공 공학부, 서울 대학교 김 승 조 2006, 11, 21 Supercomputing Korea 2006 On Computational Structures Technologies.

http://aeroguy.snu.ac.kr18/52

Commercial FEA Packages

- Parallel Module

The solvers are:- Distributed Domain Solver (DDS) - Distributed Jacobi Conjugate Gradient (DJCG) - Algebraic Multigrid (AMG) - Distributed Pre-conditioned Conjugate Gradient (DPCG)

출처 : www.ansys.com , ANSYS ver. 10.0

Page 19: IPSAP/ EXPLICIT VIBRATION ANALYSIS STRESS ANALYSIS 기계항공 공학부, 서울 대학교 김 승 조 2006, 11, 21 Supercomputing Korea 2006 On Computational Structures Technologies.

http://aeroguy.snu.ac.kr19/52

Comments– Most of commercial FEA packages use direct method : multifrontal

(ABAQUS), sparse matrix solver (NASTRAN),

Sparse Matrix, Frontal Solver (ANSYS) Commercial packages need to guarantee the end users the practicality and

reliability Practicing engineers do feel comfortable with the direct method However, the parallel performance of these packages are very poor. ANSYS is trying to add the iterative solver optionally in new version of the c

ode : an alternative method to increase the parallel performance.

Commercial FEM Packages

Page 20: IPSAP/ EXPLICIT VIBRATION ANALYSIS STRESS ANALYSIS 기계항공 공학부, 서울 대학교 김 승 조 2006, 11, 21 Supercomputing Korea 2006 On Computational Structures Technologies.

http://aeroguy.snu.ac.kr20/52

IPSAP : Internet Parallel Structural Analysis Program– General purpose FEA program– Generality, Single CPU & Parallel Performance– Written by C and C++

Performance : Faster than commercial softwares like MSC/Nastran and ABAQUS, etc

High-Performance FE Software, IPSAP

IPSAP/EXPLICIT

VIBRATION ANALYSIS

STRESS ANALYSIS

High-Performance High-Performance Parallel SoftwareParallel Software

Grand Challenge Applications (Large Grand Challenge Applications (Large Scale)Scale)

High-Performance Hardware High-Performance Hardware (Supercomputer, Clusters, GRID)(Supercomputer, Clusters, GRID)

Page 21: IPSAP/ EXPLICIT VIBRATION ANALYSIS STRESS ANALYSIS 기계항공 공학부, 서울 대학교 김 승 조 2006, 11, 21 Supercomputing Korea 2006 On Computational Structures Technologies.

http://aeroguy.snu.ac.kr21/52

High-Performance FE Software, IPSAP

IPSAP : Standard– Linear Static, Vibration Analysis : open on WEB

– Nonlinear, Thermal Analysis : under development

– FE Model : 8 node solid, 4 node solid, 4 node plane, 3 node plane, 2 node beam, Rigid body element

– Nodal force, Acceleration, Temperature load

– Multifrontal Linear Equation solver, Lanczos Eigenvalue extractor

– Library : BLAS(Basic Linear Algebra Subroutine), LAPACK(Linear Algebra Package), METIS

– MPP Parallelization

Page 22: IPSAP/ EXPLICIT VIBRATION ANALYSIS STRESS ANALYSIS 기계항공 공학부, 서울 대학교 김 승 조 2006, 11, 21 Supercomputing Korea 2006 On Computational Structures Technologies.

http://aeroguy.snu.ac.kr22/52

IPSAP : Explicit– Explicit Time Integration, Automatic Time Step Control – Elastic, Orthotropic, Elastoplastic, Johnson-Cook– EOS (Equation of State) : Polynomial Model, JWL, Grüneisen– FE Model : 8 node Hexahedron, 4 node BLT Shell

1 point integration with Hourglass Control– Object Stress Update : Jaumann rate stress update– Artificial Bulk Viscosity– Contact Treatment :

Contact Search : Bucket Sorting Master-Slave Algorithm, Penalty Method Single Surface Contact (or Self Contact)

– Element Erosion and Automatic Exterior contact surface update– MPP Parallelization

High-Performance FE Software, IPSAP

Page 23: IPSAP/ EXPLICIT VIBRATION ANALYSIS STRESS ANALYSIS 기계항공 공학부, 서울 대학교 김 승 조 2006, 11, 21 Supercomputing Korea 2006 On Computational Structures Technologies.

http://aeroguy.snu.ac.kr23/52

Procedure for modified multifrontal solver

Step1-2 Using data structure of Element connectivity

Step3-4 Element Computation is smeared in this step computed using the memory hierarchy

Optimized for finite element method

Step1.

Domain partitionin

g

Step2.

Symbolic factorizatio

n

Step3.

Numerical factorizatio

n

Step4.

Triangular solve

IPSAP – Multifrontal Solver

Page 24: IPSAP/ EXPLICIT VIBRATION ANALYSIS STRESS ANALYSIS 기계항공 공학부, 서울 대학교 김 승 조 2006, 11, 21 Supercomputing Korea 2006 On Computational Structures Technologies.

http://aeroguy.snu.ac.kr24/52

Domain partitioning with graph partitioning– Converting the FEM mesh into graph data

– Various graph building scheme WEM (Weighted Edge Mapping) WEVM (Weighted Edge and Vertex Mapping) I-WEVM (Iterative Weighted Edge and Vertex Mapping)

– Dividing the graph into k parts Graph regularity checking Exact spectral algorithm implemented State-of-the-art techniques incorporated

– METIS 4.0

– ParMETIS 3.1

IPSAP – Multifrontal Solver

Page 25: IPSAP/ EXPLICIT VIBRATION ANALYSIS STRESS ANALYSIS 기계항공 공학부, 서울 대학교 김 승 조 2006, 11, 21 Supercomputing Korea 2006 On Computational Structures Technologies.

http://aeroguy.snu.ac.kr25/52

Symbolic factorization– elimination ordering from partitioned graph(mesh)– front matrix size estimation

Core memory usage is known before real factorization Floating point operation count is estimated

1 2 5 6

3 4 7 8

9 10 13 14

11 12 15 16

1 23

4 567

Domain Order Factorization Order

IPSAP – Multifrontal Solver

Page 26: IPSAP/ EXPLICIT VIBRATION ANALYSIS STRESS ANALYSIS 기계항공 공학부, 서울 대학교 김 승 조 2006, 11, 21 Supercomputing Korea 2006 On Computational Structures Technologies.

http://aeroguy.snu.ac.kr26/52

Numerical factorization– Serial stage : domains are merged with factorized domains– Automatic Matrix assembly

– Operations are performed on dense frontal matrices

Hierarchical memory architecture of modern computers can be fully utilized

21 5 6 7 843

Elimination Tree

1 2

3

4 5

6

7

IPSAP – Multifrontal Solver

Page 27: IPSAP/ EXPLICIT VIBRATION ANALYSIS STRESS ANALYSIS 기계항공 공학부, 서울 대학교 김 승 조 2006, 11, 21 Supercomputing Korea 2006 On Computational Structures Technologies.

http://aeroguy.snu.ac.kr27/52

Parallel Stage : Distributed memory parallelization

– Merging makes the distributed frontal matrix– Factorization is performed with distributed matrixProc 0 Proc 1

Proc 2 Proc 3 Proc 2,3

Proc 0,1 Proc 0,1,2,3

FactorizationFactorization FactorizationFactorization

– 2 dimensional processor map is used

p0

p1

p2

p3

p0

p1

p2

p3

IPSAP – Multifrontal Solver

Page 28: IPSAP/ EXPLICIT VIBRATION ANALYSIS STRESS ANALYSIS 기계항공 공학부, 서울 대학교 김 승 조 2006, 11, 21 Supercomputing Korea 2006 On Computational Structures Technologies.

http://aeroguy.snu.ac.kr28/52

Parallel Stage : Distributed memory parallelization– Block cyclic distribution with 2 dimensional processor map

P 3P 3P 0P 0 P 2P 2P 1P 1

Block CyclicBlock Cyclic

– Owner of sub-matrix is dependent on ordering index– Matrix re-distribution is performed by one-to-one communication– pBLAS & SCALAPACK cannot handle variable block size

Block size is not fixed

p0

p1

p2

p3

IPSAP – Multifrontal Solver

Page 29: IPSAP/ EXPLICIT VIBRATION ANALYSIS STRESS ANALYSIS 기계항공 공학부, 서울 대학교 김 승 조 2006, 11, 21 Supercomputing Korea 2006 On Computational Structures Technologies.

http://aeroguy.snu.ac.kr29/52

Large-scale eigen analysis of 3D complex structures– Finite element method – Prediction of dynamic stability of structures– Up to millions of DOF (Degree of Freedom)– Huge-size computing/resources required– Repetition of linear equation solving

– Feasible algorithm : Block Lanczos Eigenvalue Solver

Equipped with Efficient Direct Equation Solver

IPSAP – Block-Lanczos Eigensolver

Page 30: IPSAP/ EXPLICIT VIBRATION ANALYSIS STRESS ANALYSIS 기계항공 공학부, 서울 대학교 김 승 조 2006, 11, 21 Supercomputing Korea 2006 On Computational Structures Technologies.

http://aeroguy.snu.ac.kr30/52

Block Lanczos algorithm with multifrontal solver– Use of Lanczos method with shift and inverting technique

for eigen solution

– Most of operations in Lanczos steps are required in solution procedure for (K - M)-1 and M inner product

– Efficient direct equation solver is needed for the Lanczos process

– Multiple RHS (Right Hand Side ) operation is needed for the block Lanczos process

(K - M) x= 0 (K - M)-1 Mx = x

= shift , =(shifted eigenvalue) -1

IPSAP – Block-Lanczos Eigensolver

Page 31: IPSAP/ EXPLICIT VIBRATION ANALYSIS STRESS ANALYSIS 기계항공 공학부, 서울 대학교 김 승 조 2006, 11, 21 Supercomputing Korea 2006 On Computational Structures Technologies.

http://aeroguy.snu.ac.kr31/52

Block Lanczos Iteration with MFS

Uj = MVj

(K- M)Wj = Uj

W’j = Wj - Vj-1 BTj-1

Cj = VTj MW’j

W’’j = W’j - Vj Cj

W’’j = Vj+1 Bj : QR factorization

Multifrontal Solver Effetive Mass Multiplication

GEQRF inPLASC

IPSAP – Block-Lanczos Eigensolver

Page 32: IPSAP/ EXPLICIT VIBRATION ANALYSIS STRESS ANALYSIS 기계항공 공학부, 서울 대학교 김 승 조 2006, 11, 21 Supercomputing Korea 2006 On Computational Structures Technologies.

http://aeroguy.snu.ac.kr32/52

Block version of CGS2 : classical Gram-Schmidt with reorthogonalization

0,..., 1for i n

**i jw W

0,1for k 1: 1 ( )

1: 1 1i k Ti j i j w

B V M

1: 1 ( )1: 1 1

i ki j i jw w V B

(1)i Ti j w wB M

(1)1 / ij i jw V B

(0) (1)j j j B B B

IPSAP – Block-Lanczos Eigensolver

Page 33: IPSAP/ EXPLICIT VIBRATION ANALYSIS STRESS ANALYSIS 기계항공 공학부, 서울 대학교 김 승 조 2006, 11, 21 Supercomputing Korea 2006 On Computational Structures Technologies.

http://aeroguy.snu.ac.kr33/52

ISSUE

(1) Increasing Efficiency of

Contact Force Vector Calculation

(2) Increasing Efficiency of

Internal Force Vector Calculation

na

n

nt 2/1nt 1nt 2/3nt

2/1 nt

1 nt

2/1nv

2/1n

2/1n

1nd

1nx

1n

1na

1int

nF 1ncontF 1n

extFSubroutine InternalForce

Subroutine

ContactForce

na

n

nt 2/1nt 1nt 2/3nt

2/1 nt

1 nt

2/1nv

2/1n

2/1n

1nd

1nx

1n

1na

1int

nF 1ncontF 1n

extFSubroutine InternalForce

Subroutine

ContactForce

Contact/Impact Analysis– Nonlinear explicit time integration– Contact search

IPSAP – IPSAP/Explicit

Page 34: IPSAP/ EXPLICIT VIBRATION ANALYSIS STRESS ANALYSIS 기계항공 공학부, 서울 대학교 김 승 조 2006, 11, 21 Supercomputing Korea 2006 On Computational Structures Technologies.

http://aeroguy.snu.ac.kr34/52

1. Each processor computes internal forces of own nodes

2. Commuication and addition for interface nodes (Swap,Add)

- unstructured efficient communication is implemented

1 2 3

4 5 6

7 8 9

Parallelization of Internal Force

IPSAP – IPSAP/Explicit

Page 35: IPSAP/ EXPLICIT VIBRATION ANALYSIS STRESS ANALYSIS 기계항공 공학부, 서울 대학교 김 승 조 2006, 11, 21 Supercomputing Korea 2006 On Computational Structures Technologies.

http://aeroguy.snu.ac.kr35/52

Define 3D box

Slave node updateContact Force

computed in each processor

Communication of Contact Force

Communication of slave nodes’

coordinates

Parallelization of Contact Force

IPSAP – IPSAP/Explicit

Page 36: IPSAP/ EXPLICIT VIBRATION ANALYSIS STRESS ANALYSIS 기계항공 공학부, 서울 대학교 김 승 조 2006, 11, 21 Supercomputing Korea 2006 On Computational Structures Technologies.

http://aeroguy.snu.ac.kr36/52

Hardware Software

Unit Node

CPUIntel Xeon

2.2/2.4/2.8/3.0 GHzOS

Windows Adv. Server 2000

Redhat Linux 9.0RAM DDR ECC 3GB/6GB

HDD IDE 80GB/160GB

Compilergcc-3.3 compilerIntel 8.0 compilerVisual Studio 6.0M/B

E7500/7501 Dual M/B

Total CPU520 CPUs

(2.2/256, 2.4/112, 2.8/64, 3.0/88)

MPILAM/MPI – 7.0.6MPICH – 1.2.5.2

MPI/Pro, NT-MPICH

Total Node 260 NodesJob

schedulerOpen PBS, Condor

Total Memory/Storage

1.02 GB / 25 TBGrid

MiddlewareGlobus 2.4

Network Gigabit Ethernet : Intel NIC e1000 /Fast Ethernet - 7 NFS Server

performance1.283 Tflops (Rmax)2.5 Tflops (Rpeek)

Local Gigabit

Local Fast

NFS & Gateke

eper

External Network

Rack ( 20 Node )

Rack-20 Node & Multi Trunking (4 GB Uplink) -Nortel 380-24T (Giga) & Intel 24T (Fast)

Gigabit Ethernet- Nortel 5510-48T

Fast ethernet- Intel 24T

Performance of IPSAP

Computing Environment– PEGASUS System

Page 37: IPSAP/ EXPLICIT VIBRATION ANALYSIS STRESS ANALYSIS 기계항공 공학부, 서울 대학교 김 승 조 2006, 11, 21 Supercomputing Korea 2006 On Computational Structures Technologies.

http://aeroguy.snu.ac.kr37/52

IPSAP Stress AnalysisSerial performance comparison

with NASTRAN 70.7 and ABAQUS 6.3– 32x32x32 hexagonal elements (DOFs = 107,811)

305

112 93

331

147203

1,345

0

200

400

600

800

1000

1200

1400

1600

Alpha EV67 (667MHz) IBM Power4 1.3GHz Intel Xeon 2.4GHz (Linux)

CPU

TIM

E(S

econd

) IPSAP ABAQUS NASTRAN

2.1 Gflops

Performance of IPSAP

Page 38: IPSAP/ EXPLICIT VIBRATION ANALYSIS STRESS ANALYSIS 기계항공 공학부, 서울 대학교 김 승 조 2006, 11, 21 Supercomputing Korea 2006 On Computational Structures Technologies.

http://aeroguy.snu.ac.kr38/52

0

100

200

300

400

500

600

32x32x32(107811 DOF)

2x128x128(149769 DOF)

2x2x8192(221211 DOF)

Tim

e(se

cond

) IPSAP(CPU TIME)

IPSAP(ELAPSED TIME)

NASTRAN(CPU TIME)

NASTRAN(ELAPSED TIME)

ABAQUS(CPU TIME)

ABAQUS(ELAPES TIME)

2.6 Gflops

IPSAP Stress AnalysisSerial performance comparison

with MSC/NASTRAN 2004 and ABAQUS6.4– PC : Pentium-4(northwood), 3GHz, 1G memory

Performance of IPSAP

Page 39: IPSAP/ EXPLICIT VIBRATION ANALYSIS STRESS ANALYSIS 기계항공 공학부, 서울 대학교 김 승 조 2006, 11, 21 Supercomputing Korea 2006 On Computational Structures Technologies.

http://aeroguy.snu.ac.kr39/52

IPSAP Vibration AnalysisSerial performance comparison

with MSC/NASTRAN 2004 and ABAQUS 6.4– PC : Pentrium-4, 3GHz, 1G memory

0

500

1000

1500

2000

2500

32x32x32(107811 DOF)

2x128x128(149769 DOF)

2x2x8192(221211 DOF)

Tim

e (

second)

IPSAP (elapsed)ABAQUS (elapsed)NASTRAN (elapsed)

Performance of IPSAP

Page 40: IPSAP/ EXPLICIT VIBRATION ANALYSIS STRESS ANALYSIS 기계항공 공학부, 서울 대학교 김 승 조 2006, 11, 21 Supercomputing Korea 2006 On Computational Structures Technologies.

http://aeroguy.snu.ac.kr40/52

NASTRAN ABAQUS IPSAP

Performance of IPSAP

IPSAP Vibration Analysis : Cycloidal Blade Model– Pentium IV 3.2GHz, 2.0 GB RAM, Windows XP– Elapsed time (30 modes extracted)

IPSAP : 2855 sec, NASTRAN2004 : 3251 sec, ABAQUS 6.4 : 4870 sec

47.854Hz (1st mode) 167.09Hz (3rd mode)

48.566Hz (1st mode)175.07Hz (3rd mode)

48.015Hz(1st mode)169.29Hz (3rd mode)

Page 41: IPSAP/ EXPLICIT VIBRATION ANALYSIS STRESS ANALYSIS 기계항공 공학부, 서울 대학교 김 승 조 2006, 11, 21 Supercomputing Korea 2006 On Computational Structures Technologies.

http://aeroguy.snu.ac.kr41/52

IPSAP Stress Analysis– Scalability test in Pegasus system

– 2D Mesh topology

No. of CPUs

MeshNumber of Unknowns

Operation counts

Performance

(GFLOPS)

Scaled Speedu

p

1 100x100x1 5.65E+9 6.06E+4 1.0 1.0

4 200x200x1 5.08E+10 2.41E+5 4.1 4.1

16 400x400x1 4.69E+11 9.62E+5 12.1 12.1

64 800x800x1 4.10E+12 3.84E+6 35.1 35.1

256 1600x1600x1 3.88E+13 1.54E+7 111.3 111.3

Specification of data for 2-D scalability test and results

0 50 100 150 200 2500

50

100

150

200

250

300

Sp

ee

d-u

p

Np

Ideal IPSAP

Performance of IPSAP

Page 42: IPSAP/ EXPLICIT VIBRATION ANALYSIS STRESS ANALYSIS 기계항공 공학부, 서울 대학교 김 승 조 2006, 11, 21 Supercomputing Korea 2006 On Computational Structures Technologies.

http://aeroguy.snu.ac.kr42/52

IPSAP Stress Analysis – Scalability test in Pegasus system – 3D Mesh topology

Specification of data for 3-D scalability test and results

1 40x40x40 201,720 8.11E+11 2.1 1.0

4 50x50x50 390,150 3.06E+12 8.2 3.8

16 64x64x64 811,200 1.33E+13 25.3 11.8

64 80x80x80 1,574,640 5.05E+13 65.5 30.6

256 100x100x100 3,060,300 1.92E+14 216.0 100.9

No. ofCPUs

Performance(GFLOPS)

ScaledSpeedup

MeshNumber ofUnknowns

Operationcounts

0 50 100 150 200 2500

50

100

150

200

250

300

Spe

ed-u

p

Np

Ideal IPSAP

Performance of IPSAP

Page 43: IPSAP/ EXPLICIT VIBRATION ANALYSIS STRESS ANALYSIS 기계항공 공학부, 서울 대학교 김 승 조 2006, 11, 21 Supercomputing Korea 2006 On Computational Structures Technologies.

http://aeroguy.snu.ac.kr43/52

0 100 200 300 400 5000

100

200

300

400

500

Spee

d-up

Number of CPUs

Ideal IPSAP

1 40x40x40 201,720 3.7 1.0

4 50x50x50 390,150 15.6 4.3

16 64x64x64 811,200 48.6 13.3

64 80x80x80 1,574,640 114.0 31.1

256 100x100x100 3,060,300 350.0 95.6

512 128x128x128 6,390,144 664.0 181.4

512 160x160x160 12,442,080 770.0 210.4

No. ofCPUs

Performance(GFLOPS)

ScaledSpeedup

MeshNumber ofUnknowns

IPSAP Stress Analysis– Scalability test – Computing Environment : IBM p690+ (power 4 1.7GHz)– 3D Mesh Topology

Performance of IPSAP

Page 44: IPSAP/ EXPLICIT VIBRATION ANALYSIS STRESS ANALYSIS 기계항공 공학부, 서울 대학교 김 승 조 2006, 11, 21 Supercomputing Korea 2006 On Computational Structures Technologies.

http://aeroguy.snu.ac.kr44/52

0 1 2 3 4 5 6 7 8 90

1

2

3

4

5

6

7

8

9

Spee

d-up

Number of CPUs

Ideal ABAQUS(48x48x48) IPSAP(48x48x48) IPSAP(32x32x32)

No. of CPUs

IPSAP (D-MFS) [32x32x32]IPSAP(D-MFS) D-MFS

[48x48x48]ABAQUS [48x48x48]

(DOF=107,811) (DOF=352,947) (DOF=352,947)

Elapsed time Speedup Elapsed time Speedup Elapsed time Speedup

1 112 sec 1.0 1159 sec 1.0 1345 sec 1.0

2 57 sec 2.0 585 sec 2.0 710 sec 1.9

4 32 sec 3.5 306 sec 3.8 485 sec 2.8

8 17 sec 6.6 152 sec 7.6 468 sec 2.9

IPSAP Stress Analysis– Parallel performance comparison with ABAQUS– IBM Power4 1.3GHz system

Performance of IPSAP

Page 45: IPSAP/ EXPLICIT VIBRATION ANALYSIS STRESS ANALYSIS 기계항공 공학부, 서울 대학교 김 승 조 2006, 11, 21 Supercomputing Korea 2006 On Computational Structures Technologies.

http://aeroguy.snu.ac.kr45/52

IPSAP Vibration Analysis : Simple wing structure– Parallel performance comparison with NASTRAN– PEGASUS Cluster– Distributed memory parallel – 4 node shell elements– 0.5 million DOF

0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 180

2

4

6

8

10

12

14

16

Spee

d-U

p

Number of CPUs

Ideal IPSAP NASTRAN

CPU IPSAPNASTRA

N

1 214 sec 327 sec

2 114 sec 213 sec

4 63 sec 177 sec

8 39 sec 112 sec

16 24 sec 104 sec

Solution time only

Performance of IPSAP

Page 46: IPSAP/ EXPLICIT VIBRATION ANALYSIS STRESS ANALYSIS 기계항공 공학부, 서울 대학교 김 승 조 2006, 11, 21 Supercomputing Korea 2006 On Computational Structures Technologies.

http://aeroguy.snu.ac.kr46/52

NCPU

IPSAP/Explicit ( Ncycles = 507) LS-DYNA ( Ncycles = 457)

Grind Time(nano sec) Speedup

ElapsedTime**

Grind Time(nano sec) Speedup

ElapsedTime**

2 2260 2.00 90.5 hr 2270 2.00 81.6 hr

4 1140 3.98 45.5 hr 1295 3.51 46.4 hr

8 575 7.90 22.9 hr 623 7.28 22.4 hr

16 287 15.8 11.5 hr 316 14.4 11.3 hr

32 147 31.3 5.8 hr 163 27.9 5.8 hr

64 73.3 62.0 2.9 hr 92 49.4 3.3 hr

IPSAP/Explicit– Taylor Impact Test– Comparison with LS-DYNA 970– PEGASUS Cluster– Distributed Memory Parallel– 10.0 Million DOF

Performance of IPSAP

Page 47: IPSAP/ EXPLICIT VIBRATION ANALYSIS STRESS ANALYSIS 기계항공 공학부, 서울 대학교 김 승 조 2006, 11, 21 Supercomputing Korea 2006 On Computational Structures Technologies.

http://aeroguy.snu.ac.kr47/52

Mild Steel Sphere :

diameter : 6.35mm

mass : 1.04g

Mild Steel Plate : 50mm x 40mm

thickness : 1.5 mm

Impact Velocity : 610m/s@60degree

Termination Time : 50 micro seconds

Material Model : Johnson-Cook

IPSAP/Explicit– Oblique Impact of Metal Sphere– PEGASUS Cluster– Distributed Memory Parallel

Performance of IPSAP

1 2 4 8 16 32 64 1281

2

4

8

16

32

64

128

Spe

ed U

p

NCPUs

FE+FE comm. Contact Total Ideal

Sphere : 21,600 elementsPlate : 7,488 elementsTotal : 29,088 elements 33,329 nodes 99,987 DOF

Page 48: IPSAP/ EXPLICIT VIBRATION ANALYSIS STRESS ANALYSIS 기계항공 공학부, 서울 대학교 김 승 조 2006, 11, 21 Supercomputing Korea 2006 On Computational Structures Technologies.

http://aeroguy.snu.ac.kr48/52

홈페이지 : http://ipsap.snu.ac.kr– Modules included : Stress analysis, vibration analysis– Elements : solid, shell, beam– Downloadable IPSAP executables

Windows, Linux, OS-X Serial, parallel version

released IPSAP

Page 49: IPSAP/ EXPLICIT VIBRATION ANALYSIS STRESS ANALYSIS 기계항공 공학부, 서울 대학교 김 승 조 2006, 11, 21 Supercomputing Korea 2006 On Computational Structures Technologies.

http://aeroguy.snu.ac.kr49/52

AIRBUS – Aircraft Virtual Structural Testing– 항공기 전기체 가상구조시험

Virtual Design Development

에어버스 가상구조시험 체계 계획 (2005 년 8 월 ) 가상구조시험의 멀티 스케일 해석 접근

Page 50: IPSAP/ EXPLICIT VIBRATION ANALYSIS STRESS ANALYSIS 기계항공 공학부, 서울 대학교 김 승 조 2006, 11, 21 Supercomputing Korea 2006 On Computational Structures Technologies.

http://aeroguy.snu.ac.kr50/52

Virtual Design Development

Boeing– “Virtual Mockup facilities the exploration of a larger design

solution space, at the same time that it helps catch problems before they become very expensive. This enabled Boeing’s 777 program to achieve unprecedented levels of rework reduction, product quality and customer satisfaction.”

William A. McNeelySenior Principal Scientist

Boeing Information and Support Services

Boeing 777Virtual Reality Application

Page 51: IPSAP/ EXPLICIT VIBRATION ANALYSIS STRESS ANALYSIS 기계항공 공학부, 서울 대학교 김 승 조 2006, 11, 21 Supercomputing Korea 2006 On Computational Structures Technologies.

http://aeroguy.snu.ac.kr51/52

NASA– 항공우주비행체의 개발에 걸리는 시간과 비용의 절감 및 안전성이

높은 비행체를 개발 ATED(Analytical Tools and Environments for Design) IITS(Integrated Instrumentation and Testing Systems)

– NASA 내에서 보유하고 있는 항공우주 비행체 설계에 관한 여러 가지 분야의 개발 프로그램들이 연동을 위한 환경 제공

Virtual Design Development

ATED 연구팀의 최적 항공기 설계를 위한 개념도

Page 52: IPSAP/ EXPLICIT VIBRATION ANALYSIS STRESS ANALYSIS 기계항공 공학부, 서울 대학교 김 승 조 2006, 11, 21 Supercomputing Korea 2006 On Computational Structures Technologies.

http://aeroguy.snu.ac.kr52/52

Development of Large Scale Parallel Structural Analysis Code

– SALINAS in USA

– ADVENTURE, GeoFEM in Japan Development and Improvement of High-Performance and General Purpose Structural Analy

sis Program, IPSAP– Multi-frontal Solver (Stress Analysis)

– Block-Lanczos Solver (Vibration Analysis)

– Nonlinear Explicit Time-Integration Solver (IPSAP/Explicit)

– Better Serial/Parallel Computing Performance, Compared with Commercial FE Softwares

Virtual Design and Development is already part of Structure Analysis Culture.

– Significant achievements already performed in several Industries

– Continuous improvement of Hardware and Software performance

Conclusion