Z-Push 2.0For administrators

Z-Push

• Z-Push 1.x code base 5 years old• Development & improvement of

1.5.x for 1,5 years• Considered very stable• Generally good support of devices• Supports ActiveSync 2.5

(comparable with MSEX 2003)– Early days of mobility – Only plaintext email possible

(with the exception of iOS)

Current situation: Z-Push 1.5.10

• Re-creation of the ActiveSync profiles of the users necessary in many cases

• Synchronization states are hardly manageable– Broken/missing states cause duplicates on mobile

• No effective error detection• Invalid mapi data “drives devices crazy”

– Battery drain– ‘incoming loops’ (same item created various times on the server)– Never ending synchronisations

• Many concurrent push connections (especially from iOS)• Code extensibility limited

Not optimal ‚things‘ with Z-Push 1.5

• More control– About connected users and devices– Remote-wipe even with changed User password or without store

• Improved logging– Different log levels– Ability to write full logs for certain users– Fail2ban support

• Support friendly– Generally no need to recreate profiles on the mobiles

Key features for the administrator

• Semantic checks on dataEvery data object is checked before it is sent to the mobile,

e.g. “is the end date of an appointment before the start date”

Object can be ignored if a critical criteria is not met.

Results in [WARN] level log lines and are also in the device details:

SyncObject->Check(): object from type SyncContact: parameter 'picture' is longer than 49152. Check failed (4)

• Incoming loop detection (failstate)Assures that created items on the mobile are created ONCE on the server.

Does not generate a specific log entry

Stability improvements I

Loop detection– Detects when mobiles do not understand received data. The amount of

data is reduced to 1 object trying to identify which object is causing the issue.

– When problematic item is identified it is ignored. This is logged on [WARN] log level. This message is visible in z-push-admin.

– Users could get error messages on the mobile

“Synchronization not possible at the moment”.

Retrying several time (refresh) generally solves this.

– In some cases the “Mail app” (iOS) needs to be stopped and restarted. Device reboot is also a good option.

Stability improvements II

Detection of loop in different processes• Interprocess loop detection

Example:1. Sync on an invalid folder -> triggers a FolderSync

2. FolderSync does not solve the issue

3. Sync on the invalid folder is issued again -> loops

After 2 retries a full re-sync is executed automatically

• Segfault detectionProcess did not terminate successfully.

Logs message in [ERROR] log level:LoopDetection->ProcessLoopDetectionPreviousConnectionFailed():

Command 'Sync' at 07/06/2012 17:03:51 with pid '4986' terminated unexpectedly or is still running.

Stability improvements III

• In combination with ZCP 7.0.6+ real notifications are available by using a “changes sink”!

Polling vs. Notification

Real push notifications

ZCP Z-Push

Z-Push checks (1) with the server every X seconds (2)if changes are available

ZCP Z-Push

Z-Push registers with the server on which folders it wants to be

notified on changes (3).

On change, server notifies Z-Push (4).

3

41 2

• ZCP 7.0.6+ implements changes sink (see “Sink ...”)• Fallback method is polling (says “Polling ...”)

New tools – z-push-top (live statistics)

• Several actions available– Type “h” or “help” in z-push-admin for full list– Possible to grep and tail directly on the log from tool– Filter listed output (option “f”) to only see certain commands, user, pid or

device– Activate “wide” view for large screens (option “w”)

• Hint: You can create a symbolic link on your server to easily access z-push-top and z-push-adminln –s /usr/share/z-push/z-push-top.php /usr/sbin/z-push-topln –s /usr/share/z-push/z-push-admin.php /usr/sbin/z-push-admin

Note: to execute these commands you need reading and writing permissions to the logs and states

z-push-top

• User administration (details about connected devices, remote-wipe, resynchronization)

• Administrative clean up (reset loop detection)

Device Management with z-push-admin

• Also available over SOAP interface

New tools – z-push-admin (II)

With z-push-admin• See which users have connected an ActiveSync device:

-a list

• See details about one or all devices: -a list –u USERNAME –d DEVICE

• Remote wipe a device-a wipe –u USERNAME –d DEVICE

• Remove states of all or a single devices of a user-a remove –u USERNAME [-d DEVICE]

Note: if devices are still connected they will re-provision and re-sync

• Trigger re-sync of a device-a remove –u USERNAME [-d DEVICE]

Note: remove and re-sync are similar, but no re-provision is necessary and dates like “first synchronization” stays intact.

Device management

Multiple folders per request (device decision)• Z-Push 1.5: one sync request is always for one folder• Z-Push 2.0: one sync request can be for one or multiple folders

“Partial” and “empty” sync• Partial: request sync for one or more folders, but also expects

changes for “all known synchronized folders”

• Empty: request sync without sending data, which means:

give changes for “all known synchronized folders”

Partial & Empty Synchronization

• Is a combined PING and SYNC request• Initializes like a normal sync but blocks on requested folders if there

are no changes atm• On change:

the new data is sent immediately in the response• Partial/Empty & Heartbeat can be combined

• Much lower latency till change is on the mobile– one response instead of one response and one full sync request

• Less bandwidth consumption• May look confusing in z-push-top, as command “Sync”

Heartbeat Synchronization

Real notifications & heartbeat functionality

“BlackBerry Experience”

with ActiveSync

Currently supported by

Android 4 and

Windows Phone 7+

• Timezone configuration is important for Z-Push 2– If no timezone is set (default), Europe/Amsterdam is used

• State directory:

STATE_DIR /var/lib/z-push/• Provisioning settings (as in 1.5.x)

PROVISIONING and LOOSE_PROVISIONING• Amount of items to be synchronized (default all)

SYNC_FILTERTIME_MAX SYNC_FILTERTYPE_ALL

Configuration options

• Log directory:

LOGFILEDIR /var/log/z-push/• Log level (default)

LOGLEVEL LOGLEVEL_INFO• Available:

LOGLEVEL_OFF, LOGLEVEL_FATAL, LOGLEVEL_ERRORLOGLEVEL_WARN, LOGLEVEL_INFO, LOGLEVEL_DEBUG

Containing user data:

LOGLEVEL_WBXML wbxml data (for support!)

LOGLEVEL_DEVICEID as wbxml but with device id

LOGLEVEL_WBXMLSTACK wbxml processing stack

Configuration options – Logging options

• Separate log for certain users

LOGUSERLEVEL LOGLEVEL_DEVICEID

Users which should be logged are named:

$specialLogUsers = array(‘mo’, ‘j@d.com’);

• Fail2ban log:

LOGAUTHFAIL true

Writes [WARN] message into the default log

Configuration options – Advanced logging

• Before synchronizing a folder, Z-Push will check if user has at least “Secretary permissions” on this folder

• Globally configured for all users on system $additionalFolders = array(

array( 'store' => "SYSTEM", 'folderid' => "7bd2229f7df92680b0700000000", 'name' => “Learn as SPAM", 'type' => SYNC_FOLDER_TYPE_USER_MAIL, ),

• Several folders of several stores can be synchronized• Data can be discovered using a script available for ZCP in

z-push/backends/zarafa/listfolders.php

Configuration: synchronize Public folder

• Z-Push 2 states are by default located in/var/lib/z-push

• Directory needs to be writeable for apache• Below this directory, several subdirectories are created• States can be deleted manually!

– same operation which z-push-admin resync/remove options do!

• States are located by device using the last 2 characters of the device id, e.g.

Device id States location

Appl789789Z7G /var/lib/z-push/g/7• State files begin with the device id, e.g.

Appl789789Z7G-74cad955-d390-4f02-b4a1-feb576f02a46-<xx>

State management

• The states have a fixed structure

<device-id>-<sync-uuid>-[type]-<counter>• Structure in detail

– Appl789789Z7G the device id– devicedata main device information– 74cad955-d390-feb576f02a46 Unique key for a folder (uuid)– <no-type> normal synchronization state– hc HierarchyCache– fd FolderData (used e.g. for ping)– fs FailSave data (used by incoming

loop protection)– bs Backend storage (IMAP backend)

States II

• concurrent PING connections– iOS opens a new ping connection every time an ActiveSync item is

accessed– These connections run up to 1 hour on the server and block a apache

process/thread and memory each– Each ping connection registers an own sink or worse: does polling

• PingTracking solves this– Active PING connections check regularly if newer ping processes are

available. If so, the older connections terminates.– This is done every refresh cycle

(default 30 seconds - parameter PING_INTERVAL in config.php)– Only one ping connection stays active

Ping Tracking

Files Blanks Comments Code Total

Z-Push 1.5 42 2542 4099 13549 20190

Z-Push 2.0 147 6343 19198 30237 55778

Difference + 350 % + 249 % + 468 % + 223 % + 276 %

Some stats (LoC)

Ohloh.comSummary

https://www.ohloh.net/p/z-push

• States are incompatible– Old states should be removed or not be copied– All the devices need to fully resynchronize– May cause some higher server load– May take some time (amount of users + items)

• All data is automatically removed from the mobile• Data is then resynchronized• Some users will see errors

– Device sometimes needs “time” and several sync cycles– Users should press “refresh” repeatedly– On iOS restart mail app or reboot device helps

Upgrading 1.5 -> 2.0

• Install & configure z-push 2– Set timezone– Check/Create state and log folders– State and log folders need to be writeable for apache

• Add dummy configuration to apache for testing & reload• Test in browser

– Check for errors in /var/log/z-push/z-push.log

• Update apache config & reload• Check progress with z-push-top

Upgrading steps

• Upgrade• z-push-admin• z-push-top

Demos

• Missing writing permissions to state & log files– Synchronization will not occur, errors will be in log file

• Missing packages– Z-Push 2 uses shared memory for some operations– Some OS like RHEL need package php-process– Z-push-top requires php5-pcntl on some systems, e.g. SUSE

• Appointments are listed with wrong time on mobiles– Z-Push server timezone is not set– Timezone on mobile is not set or wrong

• High latency– If ZCP server is “too far away” connections can time out

What usually goes wrong

• Z-Push 2.0 released today !

Give it a try!

Next step ?

Better/More• Control• Flexibility• Manageability• Compatibility• Performance

New• Features• Tools• Supported devices• User friendlier (profiles)

• Complementary support until 31.12.2012• Upgrade support afterwards

• Be aware that with Z-Push 1.5 more BYOD

incompatibilities

are to be expected

• Currently Android 4 &

BlackBerry Playbook

are already not

supported with 1.5.x

Z-Push 1.5 Support

• MDM plugin for Webapp with new features– More synchronization status information– Messages/objects which could not be synchronized– Open shared folders (contacts & calendars only on iOS)

• Planning 2.x:– Encrypted & signed emails – Auto-discovery– Policy management tool

• Packaging– Easier installation & upgrade

Where does it go?

• Announcements on Zarafa Release & Beta list

Subscribe at http://zarafa.com/newsletter

• Public Bug tracker: http://z-push.sf.net/tracker

• Latest downloads: http://z-push.sf.net/download

(beta & finals)

• Sources at BerliOS: https://svn.berlios.de/wsvn/z-push/

• Compatibility list: http://z-push.sf.net/compatibility

Always up-to-date list about known devices and potential compatibility issues.

Keep in touch

Questions

Thank you for your attention!