Let’s Think about Type Hints!

Yusuke Miyazaki @ymyzk PyCon JP 2016 2016/9/22 Room 203

Introduction

Introduction

• / Yusuke Miyazaki / @ymyzk

• Master’s course student of Kyoto University

• Programming Language / Type System

• Co-founder Unimap, Inc. / KyodaiMap

• Member of Student Community CAMPHOR-

• Visit ymyzk.com for more details!

Python and Me• Using for about 5 years

• Web applications (Django, Flask, Bottle…), tools, etc…

• Contributing to OSS

• Libraries (django-channels, python-gyazo, etc…)

• Translation (Python, Django)

• Local staff of Python Boot Camp in Kyoto

• Attendee of PyCon APAC 2013, PyCon JP 2014-2015

Types and Me

Python

Swift CJava

Objective-C

OCaml

Scheme JavaScript

Ruby

PHP

Types and Me

Python

Swift

C

JavaObjective-C

StaticallyTyped

Dynamically Typed

OCaml

Scheme

JavaScriptRuby

PHP

What is Type Hints?

Standardized Type Annotations

for Python

Example without Type Hints

def add(x, y): return x + y

def greeting(name): return "Hello, {}!".format(name)

def make_pair(e): return e, e

Example with Type Hints

def add(x: int, y: int) -> int: return x + y

def greeting(name: str) -> str: return "Hello, {}!".format(name)

from typing import Tuple, TypeVar T = TypeVar("T")def make_pair(e: T) -> Tuple[T, T]: return e, e

Goal of the Session

Get to know type hints

Goal of the Session

Get to know type hints

Use type hints in your projects!

Agenda

• Background

• Introduction to type hints

• Tools

• Comparison with other language

• Summary

Background

Type Checking

Type Checking

• Python is a dynamically typed language

• Fast development / prototyping

• We want to avoid runtime type errors

• Big projects

• Scientific computing / machine learning

Existing Projects

• mypy ― Optional static type checker

• PyCharm ― IDE: Use type information for completion and error detection

• Reticulated Python ― Static type checker & runtime type checking

Function Annotations

Function Annotations

• Introduced in Python 3.0 (PEP 3107)

• Annotate function’s parameters and return values with expressions

• Runtime access via __annotations__

• Expected use cases in PEP:Type information, documentation, etc…

Examples

# Documentationdef compile(source: "source code", file: "filename") -> "result": pass

# Type informationdef compile(source: str, file: str) -> str: pass

Function annotation (parameter)

Function annotation (return value)

Documentation

Documentation

• Type information as a part of documentation

• Good practice for libraries or big projects

• Write parameter types or return type in docstrings or comments

• Existing projects: Sphinx, Doxygen, etc…

Sphinx

def greeting(name): """Generate a greeting

:param name: name of someone :type name: str :return: generated message :rtype: str """ return "Hello, {}!".format(name)

Sphinx

Gradual Typing

Gradual Typing

• Type system proposed by Siek and Taha (2006)

• Statically typed parts & dynamically typed parts in a single program

• References:

• PEP 483 ― The Theory of Type Hints

• What is Gradual Typing ( )

Type Hints

Type Hints

Standardized type annotation for Python

• Introduced in Python 3.5 (2015)

• Specification: PEP 484 ― Type Hints ( )

• Use function annotation to annotate types

• Use typing module for expressing types

How to Write Type Hints?

def greeting(name: str) -> str: return "Hello, {}!".format(name)

How to Write Type Hints?

def greeting(name: str) -> str: return "Hello, {}!".format(name)

Use function annotation

How to Write Type Hints?

def greeting(name: str) -> str: return "Hello, {}!".format(name)from typing import List

Use function annotation

Import "List" to express type hints

How to Write Type Hints?

def greeting(name: str) -> str: return "Hello, {}!".format(name)from typing import List

def count_zero(elements: List[int]) -> int: return sum(1 for e in elements if e == 0)

Use function annotation

Import "List" to express type hints

How to Write Type Hints?

# Before Python 3.6 YEAR = 2016 # type: int

How to Write Type Hints?

# Before Python 3.6 YEAR = 2016 # type: int Type comments in PEP 484

How to Write Type Hints?

# Before Python 3.6 YEAR = 2016 # type: int

# NEW in Python 3.6 YEAR: int = 2016

Type comments in PEP 484

Variable annotations in PEP 526

Another Way: Stub Files

# greet.py def greeting(name): return "Hello, {}!".format(name)

# greet.pyi # A stub file for greet.py def greeting(name: str) -> str: ...

Types Defined in PEP 484

• Callable(e.g., Callable[[int, int], float])

• TypeVar (e.g., T = TypeVar("T"))

• Generics (e.g., Sequence[T])

• Any, Optional, Union, Iterable, etc…

Features of Type Hints

• Use the same type hints in various tools

• Type checker, editors, documentation, etc…

• Not required (use of type hints is optional)

• CPython ignores type hints at runtime

• Not performance boosting (at least in CPython)

• Backwards compatible

Backwards Compatibility

• typing module is just a single Python file!

• A backport is available for Python 2.7, 3.x:$ pip install typing

• Syntax for Python 2.7 is suggested in PEP:

def greeting(name): # type: (str) -> str return "Hello, {}!".format(name)

Tools

mypy

mypy

• Optional static type checker for Python

• PEP 484 is strongly inspired by mypy

• Still in development

• You can use it as a lint tool like pylint / flake8

Example

def greeting(name: str) -> str: return "Hello, {}!".format(name)

greeting("OK") greeting(1234)

Example

def greeting(name: str) -> str: return "Hello, {}!".format(name)

greeting("OK") greeting(1234)

error: Argument 1 to "greeting" has incompatible type "int"; expected "str"

typeshed

typeshed

• Repository of stub files for:

• Python builtins / standard libraries

• Third-party libraries

• A few third-party libraries are supported

• Let’s make PRs: github.com/python/typeshed

PyCharm

PyCharm

• IDE for Python

• Type checking / Code completion

• References:

• Python 3.5 type hinting in PyCharm 5

• Type Hinting in PyCharm

Example 1

Type checking

Example 2

Code completion

Sphinx

Sphinx

• Documentation tool written in Python

• Supports type annotations in docstrings

• Supports type hints too!

Examplefrom typing import Iterable def greeting_all(names: Iterable[str]) -> str: return "Hello, {}!".format(", ".join(names))

Examplefrom typing import Iterable def greeting_all(names: Iterable[str]) -> str: return "Hello, {}!".format(", ".join(names))

Comparisonwith Other Languages

JavaScript

JavaScript + Types

There are a lot of projects…

• TypeScript

• Flow

• Closure Compiler

• JSDoc

TypeScript

• Superset of JavaScript by Microsoft

• Static type checker + compiler to JavaScript

• DefinitelyTyped (corresponds to typeshed)

• Type definitions for more than 1,000 libraries

• Structural type system

Example// greet.ts const greeting = (name: string) => "Hi, " + name; greeting("Taro"); greeting(10);

Example// greet.ts const greeting = (name: string) => "Hi, " + name; greeting("Taro"); greeting(10);

// greet.js var greeting = function (name) { return "Hi, " + name; }; greeting("Taro"); greeting(10);

Example// greet.ts const greeting = (name: string) => "Hi, " + name; greeting("Taro"); greeting(10);

// greet.js var greeting = function (name) { return "Hi, " + name; }; greeting("Taro"); greeting(10);

error TS2345: Argument of type 'number' is not assignable to parameter of type 'string'.

Flow

• Static type checker for JavaScript

• Can be used without type annotations by using type inference

• You can write type annotations

• Strip type annotations to run programs

Example

// @flow function times10(n) { return n * 10; }

times10(100); times10("Hello");

Example

// @flow function times10(n) { return n * 10; }

times10(100); times10("Hello");

7: times10("Hello"); ^^^^^^^^^^^^^^^^ function call 3: return n * 10; ^ string. This type is incompatible with 3: return n * 10; ^^^^^^ number

PHP

Hack

• PHP + various features (types, collections, lambdas…)

• Static typechecker

• Runs on HHVM

• Use type information at runtime & speed up!

Example

<?php function greeting($name) { return "Hello, " . $name; } function main() { greeting("Taro"); greeting(12345); } main();

Example

<?hh function greeting(string $name): string { return "Hello, " . $name; } function main(): void { greeting("Taro"); greeting(12345); } main();

Example

<?hh function greeting(string $name): string { return "Hello, " . $name; } function main(): void { greeting("Taro"); greeting(12345); } main();greet.hh:7:14,18: Invalid argument (Typing[4110])

greet.hh:2:19,24: This is a string greet.hh:7:14,18: It is incompatible with an int

Summary of Other Languages

• There are lots of projects related to type annotations

• TypeScript is widely used & has a lot of users

• No standardized type annotations

• Rewrite type annotations to use different tools

• Not backwards compatible

• Requires new compiler / interpreter

Summary

Summary of Type Hints

• Standardized type annotation in Python

• Introduced in Python 3.5

• Use function annotation to write types

• Use typing module for expressing types

Pros of Type Hints

• Type annotation is standardized!

• Use the same annotation with various tools: mypy, PyCharm, Sphinx, etc …

• Backwards compatible (Python 2 & 3)

• No compilation or translation is required

Cons of Type Hints

• Not written by many developers

• Few stub files for third party libraries

• Not performance boosting

• Not advanced type system

Let’s Use It Now!If you are interested in type hints:

• Create tools which use type hints

• Write type hints in your project

• Write stub files for existing projects

If you are not interested in type hints:

• Tools like PyCharm will help you!

Thank You for Listening!!

• Feel free to ask me questions

• This slide will be uploaded later (see Twitter)

• Development sprint about type hints on Saturday (9/24)