MapR と Vertica エンジニアが語る、なぜその組み合わせが最高なのか? - db...

30
© 2014 MapR Technologies 1 ® © 2014 MapR Technologies 草薙 昭彦 (MapR Technologies) 相澤 恵奏 (HP) MapR Vertica エンジニアが語る、 なぜその組み合わせが最なのか?

description

MapRとVertica、なぜその組み合わせが最高なのか、実際にPOC、ベンチマークを担当する両社のエンジニアがその熱い思いをデモを交えながら解説します。MapRとVerticaを組み合わせて構築された事例を含め、HadoopとRDBMSが今どのように共存しているのか、また今後どのような形が最適なのかをエンジニアの視点から語ります。2014年6月18〜20日に開催されたdb tech showcase 大阪 2014での講演資料です。

Transcript of MapR と Vertica エンジニアが語る、なぜその組み合わせが最高なのか? - db...

  • 1. 2014 MapR Technologies 1 2014 MapR Technologies (MapR Technologies) (HP) MapR Vertica

2. 2014 MapR Technologies 2 Vertica: Vertica + Hadoop MapR: Hadoop Vertica on MapR 3. 2014 MapR Technologies 3 2014 MapR Technologies Vertica: 4. 2014 MapR Technologies 4 DB 2005 Michael Stonebraker Postgres, C-Store 2011/3/22 HP 2012/12/1 HP POSTGRES Object DB C-store Column- oriented DB 1970 1980 2000 2010 INGRES RDBMS Sybase SQL Server Aster Data Greenplum Netezza Michael Stonebraker) Vertica HPVertica 5. 2014 MapR Technologies 5 DWH DWH IBM Netezza Teradata Oracle Exadata etc OLTP RDBMS OLTPRDBMS Oracle DB MySQL Postgresetc RDBMS 6. 2014 MapR Technologies 6 Vertica (WW2500 7. 2014 MapR Technologies 7 http://www.vertica.com/2013/12/12/welcoming-facebook-to-the-growing-family-of-hp-vertica-customers/ 8. 2014 MapR Technologies 8 2014 MapR Technologies Vertica + Hadoop 9. 2014 MapR Technologies 9 SQL Vertica Hadoop 10. 2014 MapR Technologies 10 Social media IT/OT ImagesAudioVideo Transactional dataMobile Search engineEmail Texts Documents SQLRC++Java UserDefineFunction HDFS Connector HCatalog SQL Hadoop MR Hadoop Vertica 11. 2014 MapR Technologies 11 4 4 155 Hadoop Vertica ODBC MPP 50 613TByte1.4Tbyte 3PC HP Vertica 12. 2014 MapR Technologies 12 Vertica Hadoop Map/Reduce Vertica Hadoop Hadoop SQL Vertica Hadoop 13. 2014 MapR Technologies 13 MapR RDBMS + 14. 2014 MapR Technologies 14 2014 MapR Technologies MapR: Hadoop 15. 2014 MapR Technologies 15 500 MapR: Hadoop 16. 2014 MapR Technologies 16 Hadoop Open Source Open Source A C Open Source 17. 2014 MapR Technologies 17 HDFS (Read/Write) MapR Distribution for Apache Hadoop YARN Pig Cascading Spark Spark Streaming Storm* HBase Solr NoSQL Juju Savannah* Mahout MLLib GraphX MapReduce v1 & v2 APACHE HADOOP OSS Tez* Accumulo* Hive Impala Shark Drill* SQL Sentry* Oozie ZooKeeperSqoop Knox* WhirrFalcon*Flume HttpFS Hue * 18. 2014 MapR Technologies 18 MapR Distribution for Apache Hadoop YARN Pig Cascading Spark Spark Streaming Storm* HBase Solr NoSQL Juju Savannah* Mahout MLLib GraphX MapReduce v1 & v2 APACHE HADOOP OSS Tez* Accumulo* Hive Impala Shark Drill* SQL Sentry* Oozie ZooKeeperSqoop Knox* WhirrFalcon*Flume HttpFS Hue MapR Data Platform (Read/Write) MapR-FS (POSIX) MapR-DB ( NoSQL) NFS HDFSAPI HBaseAPI JSONAPI MapRControlSystem () CLIGUIRESTAPI * 19. 2014 MapR Technologies 19 Hadoop : Flux7 Labs Study, 201310 Read/Write DFSIO Read DFSIO Write MB/ MB/ 20. 2014 MapR Technologies 20 2014 MapR Technologies Vertica on MapR 21. 2014 MapR Technologies 21 MapR Direct NFS Access MapR HDFS MapReduce Hadoop NFS NFS 22. 2014 MapR Technologies 22 Vertica MapR Vertica on MapR HDFS MapReduceVertica MapR MapReduce Vertica NFS 23. 2014 MapR Technologies 23 Vertica on MapR Hadoop Vertica NFS Hadoop MapR 24. 2014 MapR Technologies 24 1: MapR Vertica Vertica MapReduce SQL Vertica 25. 2014 MapR Technologies 25 2: MapR Vertica MapReduce MapReduce SQL MapR Vertica MapReduce MapReduce SQL 26. 2014 MapR Technologies 26 2014 MapR Technologies 27. 2014 MapR Technologies 27 Hadoop Intel core i3 NUC 3 16GBSSD 128GB Gigabit Ethernet + Wifi 13.3 LCD MapR 3.1.1Vertica 7.0 Do it yourself 28. 2014 MapR Technologies 28 ls Hadoop MapR NFS Vertica on MapR MapR Vertica MapReduce MapReduce SQL MapR cp mv grep cat vi diff rsync du find wc cd gzip tar tail iconv chown 29. 2014 MapR Technologies 29 Vertica MapR MPP Hadoop VM Sandbox Vertica Marketplace https://vertica.hpwsportal.com/#/Product/%7B%22productId%22%3A %221029%22%7D/Show 30. 2014 MapR Technologies 30 Q & A @mapr_japan maprjapan [email protected] MapR maprtech mapr-technologies