Plone on RelStorage

Plone on RelStorageCalvin Hendryx-Parker

PLONE CONFERENCE 2010

Saturday, October 30, 2010

PLONE CONFERENCE 2010Overview

• How does Zope store data

• Why RelStorage?

• How to implement RelStorage

• Some reasons not to use RelStorage


PLONE CONFERENCE 2010Zope StorageMethods


PLONE CONFERENCE 2010What is the ZODB?

• Object oriented database

• Used by Zope to hold everything from contentto scripts, to service configurations


PLONE CONFERENCE 2010What is Filestorage?• The default backend for Zope/ZEO

• var/filestorage/Data.fs

• Does an in-memory index at startup

• Data appends to the end

• Back-up utility is Repozo


PLONE CONFERENCE 2010What is RelStorage?

• Drop in replacement

• Replaces ZEO

• Eliminates Data.fs

• Stores pickles

• Is still ZODB

• Uses relational databases


PLONE CONFERENCE 2010Why Change?


PLONE CONFERENCE 2010Advantages of RelStorage• Existing infrastructure

• Existing expertise

• Wide range of management tools

• High availability, high performance clusters

• No in-memory index at startup

• Scales well as your database grows

• High Concurrency


PLONE CONFERENCE 2010Real World

• Major Health System Site

• Java portion of site locked up

• Restarted Serivce with Little Impact


PLONE CONFERENCE 2010Databases Supported

• MySQL 5.0.32 or better

• MySQLdb >= 1.2.2

• PostgreSQL 8.1 or better

• psycopg2

• Oracle 10g

• cx_Oracle


PLONE CONFERENCE 2010How Do I Install It?• For MySQL add the

following to buildout.cfg:

eggs = RelStorage MySQL-Python

fake-eggs = ZODB3 = 3.7.1

[instance] type mysql db plone user plone host localhost password terriblepassword


PLONE CONFERENCE 2010How Do I Install It?• For PostgreSQL add the

following to buildout.cfg:

eggs = RelStorage

fake-eggs = ZODB3 = 3.7.1 psycopg2

[instance] type postgresql dbname plone user plone host localhost password terriblepassword


PLONE CONFERENCE 2010Options• blob-dir

• shared-blob-dir

• keep-history

• replica-conf

• pack-gc

• pack-dry-run

• cache-servers


PLONE CONFERENCE 2010Available Tools• zodbconvert

• Converts from filestorage to {mysql/pgsql/oracle}

• Convert from {mysql/pgsql/oracle} to filestorage

• Convert from one Database to another (ie: mysql to oracle)

• zodbpack

• Allows concurrent connections

• Use from cron


PLONE CONFERENCE 2010zodbconvert.conf

<filestorage source> path /opt/default/zinstance/var/filestorage/Data.fs</filestorage>

<relstorage destination> <mysql> db plone user plone passwd terriblepassword </mysql></relstorage>

Converting from filestorage to MySQL


PLONE CONFERENCE 2010Running zodbconvert

$ sudo /opt/mysql/zinstance/bin/zodbconvert --clear zodbconvert.conf

Storages opened successfully.All transactions copied successfully.


PLONE CONFERENCE 2010What’s the Catch?


PLONE CONFERENCE 2010There Are Pitfalls

• Added complexity

• Additional server load

• Additional memory utilization

• The database is full of pickles, not plain text

• Your database still needs to be packed


PLONE CONFERENCE 2010What We TalkedAbout• Zope storage engines: ZEO and RelStorage

• Why you might choose RelStorage

• How to install RelStorage

• Tools for RelStorage

• Why you might not use RelStorage


PLONE CONFERENCE 2010Links

• http://pypi.python.org/pypi/RelStorage

• http://shane.willowrise.com/archives/relstorage-1-4-0b1-and-zodbshootout


http://shane.willowrise.com/archives/relstorage-1-4-0b1-and-zodbshootout

http://shane.willowrise.com/archives/relstorage-1-4-0b1-and-zodbshootout

PLONE CONFERENCE 2010Credits:• http://www.flickr.com/photos/kaplanbr/

2465380377/

• http://www.flickr.com/photos/shuttercat7/416008944/

• http://www.flickr.com/photos/juniorvelo/3267647833/

• http://www.flickr.com/photos/arthurjohnpicton/4383221264/

• http://www.flickr.com/photos/generated/3249283352/


http://www.flickr.com/photos/kaplanbr/2465380377/




http://www.flickr.com/photos/shuttercat7/416008944/




http://www.flickr.com/photos/juniorvelo/3267647833/




http://www.flickr.com/photos/arthurjohnpicton/4383221264/




http://www.flickr.com/photos/generated/3249283352/




Check out

sixfeetup.com/demos


Plone on RelStorage

Technology

Transcript of Plone on RelStorage