Fast Texture Synthesis using Fast Texture Synthesis using Tree-structured Vector Tree-structured Vector

QuantizationQuantization

Li-Yi Wei Marc Levoy

Computer Graphics GroupStanford University

IntroductionIntroduction

Texture Synthesis

Input

Result

Desirable PropertiesDesirable Properties

• Result looks like the input

• Efficient

• General

• Easy to use

• Extensible

Previous WorkPrevious Work

• Procedural Synthesis– Perlin 85, Witkin 91, Worley 96

• Statistical Feature Matching– Heeger 95, De Bonet 97, Simoncelli 98

• Markov Random Fields– Popat 93, Efros 99

OutlineOutline

• Basic algorithm

• Multi-resolution algorithm

• Acceleration

• Applications

Texture ModelTexture Model

• Textures are– local– stationary

• Model textures by – local spatial neighborhoods

Basic AlgorithmBasic Algorithm

• Exhaustively search neighborhoods

NeighborhoodNeighborhood

• Use causal neighborhoods

Causal Non-causal

Input

Noise

NeighborhoodNeighborhood

• Neighborhood size determines the quality & cost

33 55 77

99 1111 4141

423 s 528 s 739 s

1020 s 1445 s 24350 s

Multi-resolution PyramidMulti-resolution Pyramid

High resolution Low resolution

Multi-resolution Multi-resolution AlgorithmAlgorithm

BenefitBenefit

• Better image quality & faster computation

1 level55

3 levels55

1 level1111

ResultsResults

Random Oriented

Regular Semi-regular

FailuresFailures

• Non-planar structures

• Global information

ComparisonComparison

Heeger 95 De Bonet 97 Efros 99 Our method

Input

1941 secs 503 secs

12 secs

Acceleration Acceleration

• Computation bottleneck: neighborhood search

Nearest Point SearchNearest Point Search

• Treat neighborhoods as high dimensional points

1 2 3 4 5

6 7 8 9 10

11 12

Neighborhood

1 2 3 4 5 6 7 8 9 10 11 12

High dimensional point/vector

AccelerationAcceleration

• Nearest point search in high dimensions– [Nene 97]

• Cluster-based model for textures– [Popat 93]

• Tree-structured Vector Quantization– [Gersho 92]

Tree-structured Tree-structured Vector QuantizationVector Quantization

TimingTiming

• Time complexity : O(log N) instead of O(N)– 2 orders of magnitude speedup for non-trivial images

1941 secs 503 secs 12 secs

Efros 99 Full searching TSVQ

Results: Results: Brodatz TexturesBrodatz Textures

Input Exhaustive: 360 secs TSVQ: 7.5 secs

D103

D20

Application 1: Application 1: Constrained SynthesisConstrained Synthesis

?

Possible SolutionPossible Solution

• Multi-resolution blending [Burt & Adelson 83]– produce visible boundaries

Possible SolutionPossible Solution

• Original raster-scan algorithm– discontinuities at right and bottom boundaries

Possible SolutionPossible Solution

• Adaptive neighborhoods [Efros 99]– Hard to accelerate

Modifications Modifications

• Need to use a single symmetric neighborhood

• 2 pass algorithm with extrapolation

• Spiral order synthesis

ResultResult

ResultResult

• Extrapolation

??

??

ResultResult

• Image editing by texture replacement

Application 2:Application 2:Temporal TextureTemporal Texture

• Indeterminate motions both in space and time– fire, smoke, ocean waves

• How to synthesize?– extend our 2D algorithm to 3D

Temporal TextureTemporal Texture

Fire Smoke Waves

Input

Result

Future WorkFuture Work

• More general “textures”– light fields, solid textures– motion signals– displacement maps

• Real time texture synthesis

AcknowledgmentAcknowledgment

• Kris Popat• Alyosha Efros• Stanford Graphics Group• Intel, Interval, Sony

More information

http://graphics.stanford.edu/projects/texture/