2015 Pebble Developer Retreat

waf, wscript & Your Pebble App

Cherie Williams - Troublemaker

Agenda

•Why modify my wscript? •Pebble SDK & Fat PBW •What is this “waf” thing? •Why is Pebble using waf? •wscript •Why do I care about waf? •wscript Recipes •Debugging

Why modify my wscript?Making the Pebble SDK build system work for you

You can…

•Add C compiler flags

•Add custom defines for your apps (eg. #ifdef NOLOG)

•Concatenate multiple JS files or do other file manipulations at build time

You can…

•Collect & build source from a non-standard project

•Add a linter (or two!)

•Run arbitrary scripts at build time to modify resources or other files

You can…

•Add automatic testing

•Profile builds or add build visualizations

•Read in arbitrary text at build (eg. appinfo.json)

You can…

•Include an external library

•Build libraries for distribution

…the possibilities are endless!

Fine PrintRecipes + content from today’s talk are

*not* compatible with CloudPebble

Pebble SDK & Fat PBW

SDK: Capabilities Today

•Builds for up to 3 Pebble platforms

•Can build apps, as well as workers •Handles platform-specific resources •Packages a single PBW for distribution

aplite Pebble Classic Pebble Steel

basalt Pebble Time Pebble Time Steel

chalk Pebble Time Round

What is this “waf” thing?A brief introduction to the Pebble build system

Introducing waf, a Python build system

•Open source, actively maintained •Active community on Google Groups •Task parallelization & dependency handling •Language agnostic, but includes C compiler support • Integrates with Eclipse, Visual Studio and Xcode • Includes framework to build & distribute custom build systems

Why is Pebble using waf?

This guy

Why Pebble uses waf

•Efficiently handles dependencies for large, complex projects (such as Pebble firmware) •Python!! •Easy method overriding •Can package & distribute additional scripts in the SDK •Convenience modules for Logs, Node and methods for easy file substitutions

wscriptThe build script behind waf

Basic wscript (non-Pebble)

def configure(ctx): print "Configure"

def build(ctx): print "Build"

Basic wscript

$ waf configure buildSetting top to : /Users/cherie/waf-demo/defaultSetting out to : /Users/cherie/waf-demo/default/buildConfigure'configure' finished successfully (0.009s)Waf: Entering directory '/Users/cherie/waf-demo/default/build'BuildWaf: Leaving directory '/Users/cherie/waf-demo/default/build''build' finished successfully (0.009s)

Invoking waf via the pebble-tool

pebble clean waf distclean

pebble build waf configure build (-v)

pebble build (-- --other-flags) waf configure build (--other-flags)

Standard Pebble wscriptdef options(ctx): ctx.load('pebble_sdk')

def configure(ctx): ctx.load('pebble_sdk')

def build(ctx): ctx.load('pebble_sdk')

build_worker = os.path.exists('worker_src') binaries = []

for p in ctx.env.TARGET_PLATFORMS: ctx.set_env(ctx.all_envs[p]) ctx.set_group(ctx.env.PLATFORM_NAME) app_elf='{}/pebble-app.elf'.format(ctx.env.BUILD_DIR) ctx.pbl_program(source=ctx.path.ant_glob('src/**/*.c'), target=app_elf)

if build_worker: worker_elf='{}/pebble-worker.elf'.format(ctx.env.BUILD_DIR) binaries.append({'platform': p, 'app_elf': app_elf, 'worker_elf': worker_elf}) ctx.pbl_worker(source=ctx.path.ant_glob('worker_src/**/*.c'), target=worker_elf) else: binaries.append({'platform': p, 'app_elf': app_elf})

ctx.set_group('bundle') ctx.pbl_bundle(binaries=binaries, js=ctx.path.ant_glob('src/js/**/*.js'))

Read in command line options

Configure environment for build

Run build

For each HW platform: •Create app binary •Create worker binary (optional)

Bundle into PBW

Why do I care about waf?Understanding how your Pebble project is built

wscriptThe build script behind waf

Basic wscript (non-Pebble)

def configure(ctx): print "Configure"

def build(ctx): print "Build"

Glossary

Context An object for each command executed that stores all the information necessary for the command execution

BuildContext The build context holds all the information necessary for a build

Basic wscript (non-Pebble)

def configure(ctx): print "Configure"

def build(ctx): print "Build"

Modified Basic wscript

def configure(ctx): ctx.env.MESSAGE = "Hello World"

def build(ctx): pass

Glossary

Environment A group of settings and variables that are stored in a Context and that is cached between execution of commands

NOTE: A single Context can have many, arbitrary environments

build/c4che/*.pyAR = 'arm-none-eabi-gcc-ar'ARFLAGS = 'rcs'AS = 'arm-none-eabi-gcc'BINDIR = '/usr/local/bin'BUILD_DIR = 'basalt'CC = ['arm-none-eabi-gcc']CCLNK_SRC_F = []CCLNK_TGT_F = ['-o']CC_NAME = 'gcc'CC_SRC_F = []CC_TGT_F = ['-c', '-o']CC_VERSION = ('4', '8', '4')CFLAGS = ['-std=c99', '-mcpu=cortex-m3', '-mthumb', '-ffunction-sections', '-fdata-sections', '-g', '-Os', '-D_TIME_H_', '-Wall', '-Wextra', '-Werror', '-Wno-unused-parameter', '-Wno-error=unused-function', '-Wno-error=unused-variable']CFLAGS_MACBUNDLE = ['-fPIC']CFLAGS_cshlib = ['-fPIC']CPPPATH_ST = '-I%s'DEFINES = ['RELEASE', 'PBL_PLATFORM_BASALT', 'PBL_COLOR', 'PBL_RECT', 'PBL_SDK_3']DEFINES_ST = '-D%s'DEST_BINFMT = 'elf'DEST_CPU = 'arm'DEST_OS = 'darwin'INCLUDES = ['basalt']LD = 'arm-none-eabi-ld'LIBDIR = '/usr/local/lib'LIBPATH_ST = '-L%s'LIB_ST = ‘-l%s'LINKFLAGS = ['-mcpu=cortex-m3', '-mthumb', '-Wl,--gc-sections', '-Wl,--warn-common', '-Os']

LINKFLAGS_MACBUNDLE = ['-bundle', '-undefined', 'dynamic_lookup']LINKFLAGS_cshlib = ['-shared']LINKFLAGS_cstlib = ['-Wl,-Bstatic']LINK_CC = ['arm-none-eabi-gcc']MESSAGE = ‘Hello World'PBW_BIN_DIR = 'basalt'PEBBLE_SDK = '/Users/cherie/pebble-dev/PebbleSDK-dev/Pebble/basalt'PEBBLE_SDK_COMMON = '/Users/cherie/pebble-dev/PebbleSDK-dev/Pebble/common'PLATFORM = {'PBW_BIN_DIR': 'basalt', 'TAGS': ['basalt', 'color', 'rect'], 'ADDITIONAL_TEXT_LINES_FOR_PEBBLE_H': [], 'MAX_APP_BINARY_SIZE': 65536, 'MAX_RESOURCES_SIZE': 1048576, 'MAX_APP_MEMORY_SIZE': 65536, 'MAX_WORKER_MEMORY_SIZE': 10240, 'NAME': 'basalt', 'BUILD_DIR': 'basalt', 'MAX_RESOURCES_SIZE_APPSTORE': 262144, 'DEFINES': ['PBL_PLATFORM_BASALT', 'PBL_COLOR', 'PBL_RECT']}PLATFORM_NAME = 'basalt'PREFIX = '/usr/local'RPATH_ST = '-Wl,-rpath,%s'SDK_VERSION_MAJOR = 5SDK_VERSION_MINOR = 70SHLIB_MARKER = NoneSIZE = 'arm-none-eabi-size'SONAME_ST = '-Wl,-h,%s'STLIBPATH_ST = '-L%s'STLIB_MARKER = NoneSTLIB_ST = '-l%s'TARGET_PLATFORMS = [u'basalt', u'aplite']cprogram_PATTERN = '%s'cshlib_PATTERN = 'lib%s.so'cstlib_PATTERN = 'lib%s.a'macbundle_PATTERN = '%s.bundle'

Modified Basic wscript

def configure(ctx): ctx.env.MESSAGE = "Hello World"

def build(ctx): ctx(rule="echo ${MESSAGE}", source='', target='')

Glossary

Task Generator An object that handles the creation of task instances, and helps simplify the creation of ordering constraints

Task An object that represents the production of something during the build (files, in general) and may be executed in sequence or in parallel

${MESSAGE} is shorthand for ctx.env.MESSAGE

Modified Basic wscript

$ ../Pebble/waf configure buildSetting top to : /Users/cherie/waf-demo/defaultSetting out to : /Users/cherie/waf-demo/default/build'configure' finished successfully (0.002s)Waf: Entering directory '/Users/cherie/waf-demo/default/build'[1/1] echo ${MESSAGE}:Hello WorldWaf: Leaving directory '/Users/cherie/waf-demo/default/build''build' finished successfully (0.030s)

Modified Basic wscript w/Dependency

def build(ctx): ctx(rule="echo 'hello' > ${TGT}", source='', target='message.txt')

ctx(rule="cp ${SRC} ${TGT}", source='message.txt', target='copy.txt')

Glossary

Build Order The sequence in which tasks must be executed.

Dependency A dependency represents the conditions by which a task can be considered up-to-date or not, and can be explicit (dependency on file inputs & outputs) or abstract (dependency on a value). waf uses dependencies to determine whether tasks need to be run (changed checksum of source files) and the build order of tasks.

${SRC} and ${TGT} are shorthand for source and target

Pebble Project BuildLet’s see how dependencies work

Glossary

Node A data structure used to represent the filesystem. Nodes may represent files or folders. File nodes are associated with signatures, which can be hashes of the file contents (source files) or task signatures (build files)

Node != Existing File

Glossary

Command A function defined in the wscript file that is executed when its name is given on the command-line, and can be chained with other commands

Ex: waf distclean (`pebble clean`) waf configure build (`pebble build`)

Agenda

•Why modify my wscript? •Pebble SDK & Fat PBW •What is this “waf” thing? •Why is Pebble using waf? •wscript •Why do I care about waf? •wscript Recipes •Debugging

wscript Recipeshttps://developer.getpebble.com/build

Recipe #1

Add compile time flags

Create waftools/pedantic.py

def configure(ctx): ctx.env.append_value('CFLAGS', '-pedantic')

def configure(ctx): ctx.load('pebble_sdk')+ ctx.load('pedantic', tooldir='waftools')

Add to wscript

[23/35] c: src/default.c -> build/src/default.c.18.oIn file included from ../src/default.c:1:0:/Users/cherie/pebble-dev/PebbleSDK-dev/Pebble/aplite/include/pebble.h:974:33: error: ISO C does not permit named variadic macros [-Werror=variadic-macros] #define APP_LOG(level, fmt, args...) \ ^/Users/cherie/pebble-dev/PebbleSDK-dev/Pebble/aplite/include/pebble.h:1121:3: error: type of bit-field 'type' is a GCC extension [-Werror=pedantic] TupleType type:8; ^/Users/cherie/pebble-dev/PebbleSDK-dev/Pebble/aplite/include/pebble.h:1132:13: error: ISO C forbids zero-size array 'data' [-Werror=pedantic] uint8_t data[0]; ^

Recipe #2

Add build options & associated defines

Create waftools/defines.pydef options(ctx): ctx.add_option('--no-log', action='store_true', default=False, help="Mark a build to exclude app logging output")

def configure(ctx): if ctx.options.no_log: ctx.env.append_value('DEFINES', 'NOLOG')

Add to wscript for options & configurectx.load('defines', tooldir='waftools')

Add to main project.c file

#include <pebble.h>#if defined(NOLOG)#undef APP_LOG#define APP_LOG(...)#endif

Before$ pebble install --emulator aplite --logsInstalling app...App install succeeded.[22:45:45] default.c:60> Done initializing, pushed window: 0x2001a5fc[22:45:50] javascript> Ready!

$ pebble build -- --no-logs…$ pebble install --emulator aplite --logsInstalling app...App install succeeded.[22:49:34] javascript> Ready!

After

Recipe #3

Add a linter

Create waftools/linter.pydef options(ctx): ctx.add_option('--jshint', action='store_true', help="Run JSHint on the JS files in the build")

def configure(ctx): if ctx.options.jshint: try: ctx.find_program('jshint', var='JSHINT') except ctx.errors.ConfigurationError: print "jshint was not found"

def build(ctx): if ctx.env.JSHINT: ctx(rule='${JSHINT} ${SRC}', source=ctx.path.ant_glob('src/**/*.js'))

Add to wscript

def options(ctx): ctx.load('pebble_sdk')+ ctx.load('linter', tooldir='waftools')

def configure(ctx): ctx.load(‘pebble_sdk')+ ctx.load('linter', tooldir='waftools')

def build(ctx): ctx.load('pebble_sdk')+ ctx.load('linter', tooldir='waftools')

$ pebble build -- --jshint

[25/36] jshint ${SRC}: src/js/another.js src/js/pebble-js-app.js../src/js/pebble-js-app.js: line 2, col 24, Missing semicolon.

1 errorWaf: Leaving directory `/Users/cherie/pebble-apps/default/build'Build failed -> task in '' failed (exit status 2): {task 4343355600: jshint ${SRC} another.js,pebble-js-app.js -> }' jshint ../src/js/another.js ../src/js/pebble-js-app.js '

Recipe #4

Use source code from multiple folders (2.x + 3.x)

Modify wscript def build(ctx): for p in ctx.env.TARGET_PLATFORMS:- ctx.pbl_program(source=ctx.path.ant_glob('src/**/*.c'), target=app_elf)+ ctx.pbl_program(source=ctx.path.ant_glob('src/{}/**/*.c'.format(p)), target=app_elf)

if build_worker:- ctx.pbl_worker(source=ctx.path.ant_glob('worker_src/**/*.c'), target=worker_elf)+ ctx.pbl_worker(source=ctx.path.ant_glob('worker_src/{}/' '**/*.c'.format(p)), target=worker_elf)

Recipe #5

Concatenate multiple JS files

Modify wscript def build(ctx): ctx.set_group('bundle')

+ js_files = ctx.path.ant_glob('src/**/*.js')+ if js_files:+ ctx(rule='cat ${SRC} > ${TGT}', source=js_files, target='pebble-js-app.js')

- ctx.pbl_bundle(binaries=binaries, js=ctx.path.ant_glob('src/js/**/*.js'))+ ctx.pbl_bundle(binaries=binaries, js='pebble-js-app.js' if js_files else [])

A Word on Future Proofing

Sometimes the SDK default wscript will be updated, but by abstracting code out of wscript and into waf tools, it will be

much easier to maintain customizations!

DebuggingWhen your wscript changes go wrong

`pebble -v` (waf -v) for command-lines [ 6/41] c: src/concentricity.c -> build/src/concentricity.c.16.o21:27:05 runner ['arm-none-eabi-gcc', '-std=c99', '-mcpu=cortex-m3', '-mthumb', '-ffunction-sections', '-fdata-sections', '-g', '-Os', '-D_TIME_H_', '-Wall', '-Wextra', '-Werror', '-Wno-unused-parameter', '-Wno-error=unused-function', '-Wno-error=unused-variable', '-fPIE', '-I/Users/cherie/pebble-apps/pebble-dev/PebbleSDK-dev/Pebble/chalk/include', '-I/Users/cherie/pebble-dev/PebbleSDK-dev/Pebble/chalk/include', '-I/Users/cherie/pebble-apps/concentricity/build', '-I/Users/cherie/pebble-apps/concentricity', '-I/Users/cherie/pebble-apps/concentricity/build/src', '-I/Users/cherie/pebble-apps/concentricity/src', '-I/Users/cherie/pebble-apps/concentricity/build/chalk', '-I/Users/cherie/pebble-apps/concentricity/chalk', '-DRELEASE', '-DPBL_PLATFORM_CHALK', '-DPBL_COLOR', '-DPBL_ROUND', '-DPBL_SDK_3', '-D__FILE_NAME__="concentricity.c"', '../src/concentricity.c', '-c', '-o', 'src/concentricity.c.16.o']

`pebble -vvv` (waf -vvv) for complete

21:51:50 preproc reading file '/Users/cherie/pebble-apps/concentricity/build/chalk/src/resource_ids.auto.h'21:51:50 deps deps for [/Users/cherie/pebble-apps/concentricity/build/chalk/appinfo.auto.c]: [/Users/cherie/pebble-apps/concentricity/build/chalk/src/resource_ids.auto.h]; unresolved ['pebble_process_info.h']21:51:50 task task {task 4344651216: c appinfo.auto.c -> appinfo.auto.c.16.o} must run as it was never run before or the task code changed21:51:50 runner_env kw={'shell': False, 'cwd': '/Users/cherie/pebble-apps/concentricity/build', 'env': None}21:51:50 envhash d751713988987e9331980363e24189ce []

Zone Description

runner command-lines executed (same as -v)

deps implicit dependencies found

task_gen task creation & task generator method execution

action functions to execute for building the targets

env environment contents

envhash hashes of the environment objects

build build context operations, like filesystem access

preproc preprocessor execution

group groups & task generators

$ pebble build -- --zones=deps23:01:45 deps deps for [/Users/cherie/pebble-apps/concentricity/src/concentricity.c]: [/Users/cherie/pebble-apps/concentricity/src/ui.h]; unresolved ['pebble.h']

Helpful waf Links

•Pebble Recipes •https://developer.getpebble.com/build

•GitHub Project •https://github.com/waf-project/waf

•Google Group •https://groups.google.com/forum/#!forum/waf-users

•Book •https://waf.io/book/

2015 Pebble Developer Retreat

Questions?

https://developer.getpebble.com/build