JAWS-UG HPC, 22nd April, 2016

Parallel image processingwith Lambda

Takahisa ShiratoriDept. Tech. 2

Serverworks Co., Ltd.

Lambdaで並列画像処理

About Me

ExperienceMar. 2016 – Present Dept. Tech. 2 @ ServerworksApr. 2015 – Feb. 2016 Job rotation as newcomer training

@ ServerworksApr. 2012 – Mar. 2015 Ph.D student in Fluid Mechanics

Takahisa Shiratori 　白鳥 貴久Serverworks Co., Ltd.

About Serverworks Co., Ltd.

Cloud integrator specialized in AWSPremier Consulting Partner

Advanced in backbone systemsOur customers

46 in the world5 in Japan

Fluid Mechanics 流体力学

Investigate how fluids (liquids, gases etc) flow流体（液体 , 気体など）がどのように流れるのかを探る

http://www.mdte.jp/laboratory/l_train.html

http://guncanon.web.fc2.com/text/kuriki-01.htm https://www.sbd.jp/case/category/industry/electronics.shtml

PIV in Fluid Mechanics

PIV (Particle Image Velocimetry, 粒子画像流速測定法 )

PIV

How PIV derive flows?

How PIV derive flows?

How PIV derive flows?

How PIV derive flows?

Disadvantage of PIV• Long calculation time

ex) 6000 images, 1024*1024 pixel

Solution• Parallel image

processing with AWS Lambda

N

i

N

j

N

i

N

jmjimji

N

i

N

jmjimji

fg

gYYXXgfYXf

gYYXXgfYXfYXR

1 1 1 1

22

1 1

,,

,,,

Similar or Not similar…?→ Evaluate with Cross Correlation

Why Parallel processing?

Time・・・

1st Image&

2nd Image

2nd Image&

3rd Image

3rd Image&

4th Image

Serial Processing

Parallel Processing

1/(N-1) Calculation timeN: The number of image

What’s & Why AWS Lambda?

AdvantagesEasy setup

Pricing per 100ms

Loose coupling

Requests

Lambda function

S3API Gatewaycronetc…

Architecture for SINGLE processing

PreparationSaving PIV algorithm as Lambda function

Procedure1. Saving two image files on S3

2. Saving an input CSV file on S3

3. Lambda function is invoked.

4. The flow distribution is written out as an output CSV file.

5. The output CSV file is saved on S3.

1 2

3

4

5

PIV algorithm on Lambda

Architecture for PARALLEL processing

Procedure

1. Saving all image files on S3

2. Saving parent-CSV on S3

3. Lambda function is invoked.

4. Child-CSV files are generated.

12

34

5 6

7

5. Child-CSV files are saved on S3.

6. Each child-CSV files invoke Lambda function.

7. Output CSV files are saved on S3.

Good results

Error vectors will be removed in the post-processing.

Calculation Performance

Interaction?Interaction?

　 Single Parallel

　 　 1st & 2nd 2nd & 3rd 3rd & 4thMemory Size [MB] 1536Timeout [sec] 300Calculation time [sec] 112 235 234 111Max Memory Used [MB] 28 22 23 28Price [USD] 0.280 0.587 0.585 0.278

Requests

Summary

Image processing was easily parallelized with AWS Lambda.

Motion of particles, humans, vehicles etc will be quantified in shorter calculation time.

17

Thank you foryour kind attention!

Contact Us

Serverworks Co., Ltd.sales@serverworks.co.jp

+81-(0)3-5579-8029日本語 http://www.serverworks.co.jpEnglish http://www.serverworks.co.jp/en