MongoDB & NoSQL 101 2014.9.24

Jollen, Founder <jollen@jollen.org>

www.mokoversity.com

Mokoversity

認識 NoSQL 資料庫

Mokoversity

NoSQL• Not Only SQL

• Document-Oriented

• Not a replacement for SQL database

• Schema-less

• Full Text Search, Data-processing, read faster, write faster

• etc

Document-Oriented 與 JSON

Mokoversity

Step Zero$ mongo!MongoDB shell version: 2.4.2!...!> !> db!test!> show dbs!local! 0.078125GB!vcard! 0.203125GB!> use vcard!switched to db vcard!>

Cards

mongo

db

showdbs

use

認識 Collections

> use vcard!switched to db vcard!> show collections!messages!system.indexes!users!> exit!bye

Cards

show collections

exit

什麼是 Collections• Document Store (aka folder)

• In JSON and BSON format

• aka XML、MS Word、MS Excel

• Grouping documents

• collections

• Directory hierarchies

• Collections could be considered analogous to tables

Key-Value Paris• Key–Value stores use the associative

array

• The fundamental data model of document store

• In-memory document store

• Memory Cache

• Redis and etc.

Create a new Database

$ mongo!!> show dbs!local! 0.078125GB!messages! 0.203125GB!test! 0.203125GB!> use mytest!switched to db mytest!> db.dropDatabase()!{ "dropped" : "mytest", "ok" : 1 } Cards

use <new-db-name>

Save a new Document

> use mytest!switched to db mytest!> db.mytest.save({name: 'jollen'})!> db.mytest.find()!{ "_id" : ObjectId("53bf7682af97b69f131d970a"), "name" : "jollen" }!>

Cards

save()

find()⾃自動產⽣生

認識 Schema• NoSQL 資料庫不需要事先定義欄位

• Table Definition

• Document 的欄位定義稱為 Schema

• Has Schema 或 No Schema

Query a Collection• find()

• find({name: ‘Jollen’})

• findOne()

Remove a Document• remove()

• remove({name: 'jollen'})

Update a Document• update({name: 'jollen2'}, {name:

'jollen3'})

• update({name:'jollen'}, {$set: {name: 'jollen2'}}, {multi: true})

http://docs.mongodb.org/manual/reference/method/db.collection.update/

Query criteria

fields

New document !content

基本的 CRUD 操作• find()－Read

• Query Criteria

• Projection

• Aggregation

• update()－Update

• Query Criteria

學習要點• NoSQL 初學者應先學習 Collection 的操作

• CRUD

• 類似於使⽤用 SQL 語法做 Table 的查詢、新增、更新與刪除

• 學習 Schema Design 觀念與實作

• 學習 Query Criteria

• 學習 Aggregation

• MapReduce 的基礎

Query Criteria• 各式「條件查詢」的做法 • {age: {$gt: 18} }

• WHERE

http://docs.mongodb.org/manual/core/read-operations/

MongoDB CRUD

Mokoversity

Implement CRUD• MongoDB Shell

• MongoDB Script

• MongoDB Drivers

• Node.js (JavaScript)

• Python / Ruby

• C / C++

• etc - http://docs.mongodb.org/ecosystem/drivers/

MongoDB Script• 使⽤用 JavaScript 語法

• 進⾏行 CRUD 操作

• 撰寫管理功能 • 簡單的 Data Processing (MapReduce)

Create One Document// 0001!!{! var db = connect('localhost/test');!! db.vipData.save({name: 'jollen', tel: '09384567182'});!! print('0001-create-one-document finished.')!}

List Collection// 0002!!{! var db = connect('localhost/test');!! db.vipData.find().forEach(function(user) {! print("User: " + user.name + ", tel: " + user.tel);! });!! print("Info: 0002-list-collection finished.");!}!

Update Document

// 0003!!{! var db = connect('localhost/test');!! db.vipData.update({name: 'jollen'}, { $set: {name: 'jollenchen'} });!! print('0004-update-document finished');!}

Add News Field// 0004!!{! var db = connect('localhost/test');!! db.vipData.find().forEach(function(data) {! data.isActive = true;! db.vipData.save(data);! });!! print('0004-add-new-field finished');!}

Remove Document

// 0005!!{! var db = connect('localhost/test');!! db.vipData.remove({name: 'jollen'});!! print('0005-remove-document finished.');!}

Schema Design

Mokoversity

基本觀念• 類似關鍵式資料庫的 Table 設計

• Flexible Schema

Schema 設計原則• When designing data models, always consider

the application usage of the data (i.e. queries, updates, and processing of the data) as well as the inherent structure of the data itself.

• There are two tools that allow applications to represent these relationships: references and embedded documents.

• References

• Embedded Datahttp://docs.mongodb.org/manual/core/data-modeling-introduction/

Embedded DocumentPost Collection

_id

title

content

tags

Nickname

Fullname

Mobile

Age

Reference Model 使⽤用原則• 亦稱為 Normalized Data Model

• 主要⽤用途 • One-to-Many Relations*

• Many-to-Many Relations

• 主要缺點 • ⽂文件變⼤大時造成 write performance 變差

One-To-Many 的情況 #1

{! _id: "joe",! name: "Joe Bookreader”,! address:! {! patron_id: "joe",! street: "123 Fake Street",! city: "Faketon",! state: "MA",! zip: "12345"! }!}!!

{! _id: "joe",! name: "Joe Bookreader"! address:! {! patron_id: "joe",! street: "1 Some Other Street",! city: "Boston",! state: "MA",! zip: “12345"! }!}

Reference Model

http://docs.mongodb.org/manual/core/data-modeling-introduction/

關於 Reference Model• 建議使⽤用的 Data Model

• Using references between documents

• Avoid duplication of data

• Provide sufficient read performance advantage

Schema DesignUser Collection

_id

name

email

address

age Message Collection

_id

message

uid

專案描述• 將 Microsoft Excel 資料庫，轉換⾄至 NoSQL 的開發任務

• 客⼾戶資料是以 Excel 維護，現在想將客⼾戶資料「上雲端」，並製作 App 以隨時追蹤或查詢

• 將客⼾戶資料轉換為 NoSQL 資料庫。以便進⾏行 RESTful Web Service 的開發，以及 App 的製作

CRUD: Task 1• 建⽴立 vcard 資料庫

• 新增資料⾄至 vcard.users collections

• 資料欄位 • name

• phone

• email

• address

• age

提⽰示• 將 users.xls 匯⼊入 vcard.users collection

• 註：users.xls 是程式產⽣生的假資料，⾮非真實資料

CRUD: Task 2• 新增資料⾄至 vcard.messages collections

• 資料欄位 • uid（紀錄 ObjectId）

• message（⽂文章內⽂文）

Aggregation (MapReduce)

Mokoversity

使⽤用 Query Criteria

http://docs.mongodb.org/manual/core/crud-introduction/

CRUD: Task 3• 公司想針對 45 歲以下的客⼾戶進⾏行⾏行銷推廣

• 找出 18 歲以上的客⼾戶

使⽤用 Aggregationexports.query = function(req, res) {!! var model = req.app.db.models.User;!!! model!! .aggregate([! {! ! $project: { _id: 1, name: 1, age: 1 }! },! {! ! $match: {age: {$gt: 45} }! },! {! ! $match: {age: {$lt: 60} }! }])!! .exec(function(err, users) {!! ! res.send(users);!! ! res.end();!! });!}

REST API－ Using Node.js and mongoose

Mokoversity

REST API 規劃

認識 CRUD

User Collection 的 Schema// Model name: ‘User’!var userSchema = new mongoose.Schema({! name: { type: String, default: ''},! email: String,! address: String,! age: { type: Number, default: 0 }!});

User Collection

name

email

address

age

Message Collection 的 Schema

// Model name: ‘Message’!var messageSchema = new mongoose.Schema({! uid: { type: mongoose.Schema.Types.ObjectId, ref: 'User' },! message: String!});

Message Collection

_id

message

uid

使⽤用 $in Operator• 興趣有 ‘sport’ 的會員

{ interests: { $in: [ 'sport' ] } }

Populate

model! .find()! .populate('uid')! .exec(function(err, posts) {! res.send(posts);! res.end();! });