What is data engineer?資料工程師是什麼?
Transcript of What is data engineer?資料工程師是什麼?
![Page 1: What is data engineer?資料工程師是什麼?](https://reader036.fdocument.pub/reader036/viewer/2022081503/586fde8a1a28ab18428b6c07/html5/thumbnails/1.jpg)
What is Data Engineer?炬識科技股份有限公司
![Page 2: What is data engineer?資料工程師是什麼?](https://reader036.fdocument.pub/reader036/viewer/2022081503/586fde8a1a28ab18428b6c07/html5/thumbnails/2.jpg)
資料科學人才種類
資料來源:經理人月刊 137 期
2Data Engineer
![Page 3: What is data engineer?資料工程師是什麼?](https://reader036.fdocument.pub/reader036/viewer/2022081503/586fde8a1a28ab18428b6c07/html5/thumbnails/3.jpg)
資料科學領域
3
Data Engineer
![Page 4: What is data engineer?資料工程師是什麼?](https://reader036.fdocument.pub/reader036/viewer/2022081503/586fde8a1a28ab18428b6c07/html5/thumbnails/4.jpg)
2015年資科工作者約 4萬人
4
資料來源: 104 人力網 /iThome
![Page 5: What is data engineer?資料工程師是什麼?](https://reader036.fdocument.pub/reader036/viewer/2022081503/586fde8a1a28ab18428b6c07/html5/thumbnails/5.jpg)
2018年資料工程師需求 4.5萬人
5
資料來源: 104 人力網 /iThome
![Page 6: What is data engineer?資料工程師是什麼?](https://reader036.fdocument.pub/reader036/viewer/2022081503/586fde8a1a28ab18428b6c07/html5/thumbnails/6.jpg)
為何需要這麼多資料工程師?
6
資料來源:政府該做的是動手改革高教體系培養資料科學家 /TechOrange
![Page 7: What is data engineer?資料工程師是什麼?](https://reader036.fdocument.pub/reader036/viewer/2022081503/586fde8a1a28ab18428b6c07/html5/thumbnails/7.jpg)
資料工程師應具備的知識領域
7
資料來源: Data Scientist vs Data Engineer / Data Science 101
![Page 8: What is data engineer?資料工程師是什麼?](https://reader036.fdocument.pub/reader036/viewer/2022081503/586fde8a1a28ab18428b6c07/html5/thumbnails/8.jpg)
新手資料工程師應具備的技能• Hadoop-based technologies (e.g. MapReduce, Hive and Pig)• SQL-based technologies (e.g. PostgreSQL and MySQL)• NoSQL technologies (e.g. Cassandra and MongoDB)• Python, Java• Data warehousing solutions• Linux
8http://www.mastersindatascience.org/careers/data-engineer/
![Page 9: What is data engineer?資料工程師是什麼?](https://reader036.fdocument.pub/reader036/viewer/2022081503/586fde8a1a28ab18428b6c07/html5/thumbnails/9.jpg)
資深資料工程師應具備的技能• Statistical analysis and modeling• Database architectures• Hadoop-based technologies (e.g. MapReduce, Hive and Pig)• SQL-based technologies (e.g. PostgreSQL and MySQL)• NoSQL technologies (e.g. Cassandra and MongoDB)• Data modeling tools (e.g. ERWin, Enterprise Architect and Visio)• Python, C/C++ Java, Perl• MatLab, SAS, R• Data warehousing solutions• Predictive modeling, NLP and text analysis• Machine learning• Data mining• UNIX, Linux, Solaris and MS Windows
9http://www.mastersindatascience.org/careers/data-engineer/