Think sync (July 20, 2003) 同期を考えよう
-
Upload
masanari-motohasi -
Category
Technology
-
view
946 -
download
3
description
Transcript of Think sync (July 20, 2003) 同期を考えよう
Think the Synchronization!
ari (masanari Motohasi) Institute of Mobile PC Technology
Purpose
● How many PC or PDA do you have?– Office, Home, Mobiles ... any environments...– We wish that I had access one Information anywhere.
● the complete networking– However, there are the barriers by cost, security, out of
range...● Do you have backup routinely?● We need to synchronize the information!
replication system
● high availability ( 高可用性 )– redundancy ( 冗長化 ) / ritualized
● consistency / concurrency ( 一貫性 ) – contention ( 衝突回避 )
● response time / reasonable ( 高性能 ) ● easy to use / manage● warm or cold standby
ultimately ubiquitous network● インターネットにどんなところからでもアクセス
– いつでも、どこでも、どんな方法でも
– 分散されていて、もっとも正規化が進む
– 情報の同期は考えなくていいのでは?
● しかし、ネットワーク的につながっている必要がある– セキュリティとプライバシは大丈夫?
– 同じセグメントかファイアウォールを抜く必要
– コスト、圏外、公衆電話?
Viewpoints
● like network layers (vertical)– physical/hardware to applications/software
● Transparency or not (horizontal).– normalized and distributed– “use caches” have the contended problem.– “make copies (backups)” have a lag time.– recovery time is very important
● clear the “contention problem”.● hot or cold standby?
Transparency
groupware / mail[communication]
mobile PC/PDA [any]
Version Controls[file]
backup [file](Tape etc)
SAN(snapshot)
Application/Software
Database[transaction]
Physical/Hardware[unit]
Removable MediaHDD/CF/SD..[file]
file sync. (rsync/ftpsync)remote access
inet (VPN / SSH)dial up [any]
web [page/service]
SAN / NAS /Internet Disk
grid/(mobile) agent[process/object]
RAID 1|5
Directionstelnet
remote display
dump/loaddatabase
P2P
Applications / Software
Web pages and services
● one of the most distributed and normalized solution with the Internet
● standard in the world– “standard” HTML / XML / SOAP etc– very reasonable
● with secure– SSL (server|client certification, encryption)– and the other – only good design (^_^;
Web pages – Just Try It!
● say nothing, because everybody knows (^ ^;)
Web Services – Just Try It!
● SOAP / XML on HTTP– Apache SOAP (Java), Microsoft SOAP toolkit, SOAP
module(Perl)– connect to the other world.
● Internet Applications (100% pure Java) and legacy systems (Microsoft COM (ASP / VB / C)).
● “Alive Check” system for an services (SOAP).● B2B applications (XML on HTTP POST)
SOAP(HTTP+VPN)J2EE
SOAPB2B
another sitesMicrosoft COM
Remote Console, Remote Display
● Remote Console– telnet (mainly UNIX)
● Remote Display / Desktop– VNC (Virtual Network Computing) *
● Windows and Unix– pcAnywhere *
● Windows only– X
● mainly UNIX– Terminal Server (Windows)
Remote Console, Just Try It
● telnet and telnetd (Unix)● Services for Unix (Windows) 29,800yen
– functions● telnet daemon (password sync Windows to Unix)● server/client/gateway for NFS● server for NIS
– for developers● all developers telnet, compile and test the applications
Remote Display, Just Try It
● VNC (Virtual Network Computing) – for developers and tests only– check the security
● pcAnywhere– control and administer to the servers on the branches
● for security patch and maintenance● terminal service (Windows 2000)
– another profiles
Groupware / Mail
● communication!– from mails to workflow systems
● Internet Mail System (on SMTP/POP3/IMAP etc)– very simple protocol– MIME
● Exchange / Notes– has replication options– share files, meeting agents, but depend on the special
platform
Internet Mail, Just Try It!
● Everybody knows in many OS and devices– Windows, Macintosh, UNIX, PDA, Phones, JRE...– devices and nodes
● for the synchronized files– send the attached mail oneself. (MIME)– it's not useful (not seamless and automatically)
● see “mailtrek” ...– however the firewall problem is clear! (with encrypt)
● check the contents and words
Groupware, Just Try It
● Microsoft Exchange– only Microsoft world( ?NET technology)
● Louts Notes / Domino– closed technology and culture (not open).– the Intranet and workflow systems
● have rich functions– mail system, scheduler, address books, public files,
databases, replications● depend on one company ... risk!
Database Management System
● to be one of most normalized– keep isolation by “transaction”
● SQL is the standard in the world– Oracle, Sybase ASE, Postage SQL, MS SQL...
● but mainly use in the LAN. ● 2 phase commit● replication server
– transaction base.
Database, Just Try It
● Sybase Adaptive Server Enterprise 11.0 – 12.5 *● Oracle8i *● Microsoft SQL Server *● My SQL● PostgreSQL
replication – mirroring
● write the many devices as same time
controller
WAN?
controller
read/write
applications / users
databases / devices / repositories(?)
replication – copy/ snapshot
● write the transaction to the primary– copy / snapshot to the secondary (only one way)
copydump
read / write
snapshot
replication - data trigger
● the trigger – one way– performance overhead– very complexly– rollback is not easy
read/write
! data updated
write
replications – stored & forward
differencedump transaction
queue, DB, file, transaction log
2 phase commit
● 2 phase commit protocol is not redundant– just distributed
are you ready?
yes yes NO
replications – bi-directions
read/write read/write
read only read only
site A site B
read/write
site A
read/write
site B
!contention
make the contentions
● when two or more “persons” update one information as same time.– objects(Beans), file, transaction, page, law...
update!
time
A
open
B
open
update
?
contention – lock
● blocking oriented– DBM, File Systems– use the wait queue– deadlock
time
A
open
B w
ait
update
update
openlock
B (lock waits)
contention - Optimistic Concurrency Control
● restart oriented– CVS, H/A EJB
time
A
open
B
update
update
re-open
openB
×
Version Controls
● not only “source” but “documents”● CVS [Concurrent Versions System]
– optimistic concurrency control method– shell (command prompt) and GUI– over HTTP, FTP, MAIL.– ported to any OS, but mainly in UNIX– controlling my documents.
● Microsoft Visual SourceSafe– for developers
File sync / replications
● rsync● Microsoft Products
– Briefcase (Windows 95 -) *– Replication Service (Windows NT)– Offline File Synchronization (Windows 2000 -)– Microsoft ActiveSync *
● Desktop PC – Pocket PC (serial / infrared)● another products
rsync, Just Try It!
● rsync on NT (run as services)– easy?
● rsync algorithm– low-bandwidth high-latency bi-directional
communications– great– maybe not secure, not use in the naked Internet...
Microsoft Briefcase, Just Try It
● Briefcase– ブリーフケースは直接ケーブル接続またはリムーバブル ディ
スクを使用してコンピュータ間のファイル転送を頻繁に行う場合に最適なツールです。ブリーフケースを使用して、メインコンピュータ上のファイルと他のコンピュータ上で変更したファイルを同期させることができます。複数のブリーフケースを作成してファイルを整理することができます
– easy to use, but performance is no good (many files with Gateway2000 Handbook).
Microsoft Offline Files● オフラインファイルはネットワーク上の共有ファイルを操
作するのに最適なツールです。オフライン ファイルを使用すると、ネットワークに接続していないときに変更した共有ファイルを次にネットワークに接続したときに同期させることができます。
● 使ったことがある方、どんな感じでしょう?
Microsoft Directory Replication Service
● “Windows NT Core Technology” で知りましたが、 実際の運用で使用したことはありません。– 使ったことがある方、どんな感じでしょう?
Microsoft ActiveSync, Just Try It
– Desktop PC – Pocket PC (serial / infrared)– Outlook data (mails, schedules, tasks, addresses...) and
files– easy to use
● but I am used to erased all data by miss operations.
ftptrek / mailtrek, Just Try It!
● ftptrek (Java)– synchronize with the data
● faster than Windows(SMB) copy● check the size and hash of the file● support the zipped, parallel and serial copies
– synchronize with database transaction– distribute over the program file updated
● mailtrek (Java)– SOAP on SMTP/POP3/IMAP? HTTP on MIME?– Sorry, under construction
Physical / Hardware
Remote Access
● very basic technology (PPP, SLIP...)– dial-up to the node (ex. office or home)– via Internet access
● the any services on PPP– wired and seamless!
Remote Access (Internet)
● same as dial-up, but NOT secure● SSH / VPN
– need: the secure server on the Internet– need: Global IP, Static IP or Dynamic DNS– need: keep the security and the privacy
● ex. Many site that has old version OpenSSH is attacked.● ex. Do dial-up users need to open the port 23(telnet) for the
Internet?– SoftEther Virtual Ethernet System
● ex. Ethernet on the HTTPS and SSH
VPN, Just Do It
● IPsec (commercial)– CISCO VPN 3005– connected to another site
● IPsec (private)– under construction– with FTTH (B-flets, NTT)– share the files, administrate the server
SoftEther Vitual Ethernet System
● SoftEther Virtual Ethernet System● Ethernet on the HTTPS and SSH● Able to pass through the firewall● Take care for use.
mobile PC / PDA● 皆さん、いろいろな使い方を教えてください!
● notebook PC● PDA
Storages
● SAN (Storage Area Networks)– fast/virtual Access (Fiber Channel)– snapshot make the backup in a second
● Network Attached Storages● Internet Disk
– use the Virtual Disk on the Internet Servers
SAN, Just Try It
● databases and file servers– critical missions
● snapshots is great– fast access (fiber channel and huge cache)– the backups and recoveries are in a second. – but the recoveries need the time
● full backup a day, and “loading transactions” are needed.
NAS (at home), try it
● HD-HLAN, Buffalo (Link Station)– change from the file server to HLAN (NAS)– low power, small size and silent– schedule backups the file a day via “rsync”
HLAN
no redundancy160GB
Godzillafile serverRAID 520GB
(4G×6)
Internet Hard Disk, Just Try It
● accessible to the disk like a drive● test the “MYDISK” service of rimnet.
– like the disk drive on the Windows● web accessible● 100MB, free● show demo...
– no accessible through our firewall● need the authentication to proxy
– stable?
Removable Devices
● HDD, Flash Memory(CF/SD)...– use as primary devices and backups– has the risks
● if lost (stop the job?)● if stolen (get out the information?)
● encryption– Encrypting File System (EFS) on windows 2000
● over NTFS 5.0– PGPdisk
RAID 1 or 5
● HDD is one of the most fragile parts– redundancy the HDD
Backup, Just Do It
● Tape– Just standard, but very expensive for personal user– serial access– DLT / DLT / LTO...
● HDD– Random access– I use the bus-powered 2.5inch HDD with rsync (at
home)
Enterprise Disaster Recover Site,Just Do It
● for Business Continuity Planning (BCP)– Reduce the risk! plug off?
● redundant the anything– Database synchronized by ftptrek– Wires, networks (include Internet) and servers– humans and business processes
● warm standby– if “the incident” is happened, switch in a minute.
Thank you!