Aravind Sundare s an Ram a Ch e llappa Aravind Sundare s an Ram a Ch e llappa Departm ent of...
1
A CQUISITIO N OF ARTICULATED H UMAN B ODY M O DELS USING M ULTIPLE CAMERAS Arav ind Sundare san Ram a Ch e ll appa D e partm e nt of El e ctrical and Com pu te r Engine e ring, Ce nte r for A u tom ation Re s e arch Unive rs ity of M aryl and, Coll ege Park , MD 20742 USA A bstract We model th e h um an body as a set of tapered super-quadrics connected in an articulated structure and propose an algorith m to autom aticall y e stim ate th e param e te rs of th e m ode l using vide o se q ue nce s obtaine d from m ultipl e cal ibrate d cam e ras. Th e proposed m eth od first com putes a voxelrepresentation from th e im ages and m aps th e voxe l s to a h igh dim e nsional space in order to extract th e 1D structure. A bottom -up approach is th en suggested in order to build a param etric, spl ine-based representation of a general articulated body in th e h igh dim ensional space follow ed by a M ode l We model th e h um an body as com prising of several rigid body segm ents th at are conne cte d to e ach oth e r at spe cific joints form ing 1-D k ine m atic ch ains originating from th e trunk . Th e individualbody segm ents can be described by an arbitrary sh ape in term s of th e coordinate s of th is fram e , and in our case is m ode ll ed using a tapered super-quadric. O bje ctive is to e stim ate th e h um an body m ode l param eters and pose from th e voxel data using a com pl e te l y autom atic algorith m . H um an body se gm e nts and joints Conve rting binary fore ground sil h oue tte s to voxe l s using space carving. W e convert foreground sil h ouettes obtained from 12 cal ibrated cam eras into th e voxel re pre se ntation using space carving te ch niq ue s. M apping to Laplacian Eige nspace W e e xploit th e fact th at th e h um an body consists of pre dom inantl y 1D articulate d ch ains. In order to segm ent th em into different ch ains, w e m ap th e voxel s from 3D to Laplacian e ige nspace W e construct a graph w ith e ach voxe l as a node and place an edge betw een neigh bouring voxe l s to obtain a w e igh t m atrix W . W e can th e n obtain th e m apping into Laplacian e ige nspace by com puting th e e ige nve ctors of th e Laplacian of th e w eigh t m atrix, W . Th e val ue of th e i th node in th e j th dim ension is give n by th e i th el e m e nt of th e j th e ige nve ctor. S e gm e ntation in Eige nspace W e se gm e nt th e voxe l s by fitting spl ine s to th e voxe l s in e ige nspace . W e begin w ith th e voxel s th at are at th e extrem ities of th e structure in eigenspace. W e grow th e spl ine by adding voxel s th at are close to th e grow ing end of th e spl ine . W e stop grow ing th e spl ine w h en th e 1D m odel of th e spl ine does not h old, i.e., at th e junctions w h e re th re e or m ore se gm e nts m e e t (ne ck , groin). Th e voxe l s before and after segm entation are presented in th e figure above. For each node in each spl ine, y i , w e h ave th e corresponding site param eter, t i , and w e obtain a spl ine, f(.), such th at y i = f(t i ). W e th us obtain th e position of each node along th e ch ain. For each node w e al so h ave x i , th e position in norm al space. W e can use th e (t i ,x i ) pair to com pute th e sk e l e ton in norm al space for e ach se gm e nt. 3D space to Laplacian Eige nspace : Dim . 1-3, Dim . 4-6 3D space to Laplacian Eige nspace : Dim . 1-3, Dim . 4-6 R e gistration to body parts W e can register th e segm ented body parts to th e know n body m odel using probabil istic te ch niq ue s by m e asuring prope rtie s of th e spl ines such as l e ngth and th ick ne ss. W e are th us abl e to re solve am biguitie s l ik e in th e e xam pl e be low . (c) Sk e l e ton Curve (b) Se gm e nte d (e ) Conne cte d sk e l e ton (d) Conne ctivity Graph (a) Mode l E xpe rim e ntal re sults Th e algorith m w as tested on different subjects of varying BMI (body-m ass index). In each case 20 correctl y segm ented fram es w ere sel ected for estim ating h um an body m odel param e te rs. Th e re sults are pre se nte d be low . C oncl usion Transform ation to Laplacian eigenspace is very useful in order to perform segm entation of conne cte d articulate d ch ains and h as advantage s ove r oth e r m e th ods. Th e 1D structure of th e voxel s in Eigenspace can be exploited for segm entation using 1D spl ine s. Distance pre se rving transform ations such as LLE w ill not e xtract th e 1D structure . Effect of articulation is rem oved. Th e pose of th e subject does not m atter. W e are al so abl e to obtain th e position of e ach voxe l along th e articulated ch ain. Subje ct B: 165cm , 64k g Subje ct C: 172cm , 52k g Subje ct D: 178cm , 115k g Subje ct A: 184cm , 72k g E stim ating h um an body sk e l e ton m ode l Th e h um an body m odel consists of joint locations, bone l ength s, and param eters of th e super-quadrics of various body segm ents. The skel eton body m odel (joint locations and bone l ength s) can be com puted by fitting l ine segm ents to th e sk el etons estim ated in Figure (e) above. Optim isation of th ese param eters can be perform ed using 20 different fram es. Once the skel eton m odelparam eters are initial ised and optim ised using th e com puted skel etons, w e estim ate th e super quadric param eters for th e different body segm ents using th e voxe l s. (a) Afte r Initial isation Model skel e ton supe rim pose d w ith com pute d sk e l e ton for a fe w fram e s. (b) Afte r final optim isation C om puting supe r-q uadric param e te rs Super quadric param eters for each body segm ent are com puted by considering th e radial profil e of th at body segm ent for all th e fram es. Once th e pose and sk el eton param eters are obtained, th e voxel s belonging to each body segm ent can be segm ented and th eir radial profil e along th e ir principal axis plotte d. Th e radial profil e is th e th ick ne ss of th e se gm e nt along th e principal axis. Th e radial profil e for som e se gm e nts are plotte d be low . Supe r-q uadric Radial Profil e Vie w Com pute d radial profil e s for diffe re nt se gm e nts (a) Trunk (b) H e ad (c) Fore arm (d) Le g Sam pl e im age s from m ultipl e cam e ras w ith back ground subtraction Voxe l s in 3D space
Transcript of Aravind Sundare s an Ram a Ch e llappa Aravind Sundare s an Ram a Ch e llappa Departm ent of...
ACQUISITION OF ARTICULATED H UMAN BODY MODELS USING MULTIPLE CAMERAS
Aravind Sundare s an Ram a Ch e llappaD e partm e nt of Ele ctrical and Com pute r Engine e ring, Ce nte r for Autom ation Re s e arch
Unive rs ity of Maryland, Colle ge Park , MD 20742 USA
Abstract
W e m odel th e h um an body as a set of tape red supe r-q uadrics connected in an articulated structure and propose an algorith m to autom atically e stim ate th e param ete rs of th e m odel using video se q uence s obtained from m ultiple calibrated cam e ras.
Th e proposed m eth od first com pute s a voxel repre sentation from th e im age s and m aps th e voxels to a h igh dim ensional space in orde r to extract th e 1D structure.
A bottom -up approach is th en sugge sted in orde r to build a param etric, spline -based repre sentation of a gene ral articulated body in th e h igh dim ensional space follow ed by a
M ode l
W e m odel th e h um an body as com prising of seve ral rigid body segm ents th at are connected to each oth e r at specific joints form ing 1-D k inem atic ch ains originating from th e trunk .
Th e individual body segm ents can be de scribed by an arbitrary sh ape in te rm s of th e coordinate s of th is fram e , and in our case is m odelled using a tape red supe r-q uadric.
Objective is to e stim ate th e h um an body m odel param ete rs and pose from th e voxel data using a com pletely autom atic algorith m .
H um an body se gm e nts and joints Conve rting binary fore ground silh oue tte s to voxels using space carving.
W e convert foreground silh ouette s obtained from 12 calibrated cam e ras into th e voxel repre sentation using space carving tech niq ue s.
M apping to Laplacian Eige nspace
W e e xploit th e fact th at th e h um an body consists of predom inantly 1D articulated ch ains. In orde r to segm ent th em into diffe rent ch ains, w e m ap th e voxels from 3D to Laplacian e igenspace
W e construct a graph w ith each voxel as a node and place an edge betw e en ne igh bouring voxels to obtain a w e igh t m atrix W .
W e can th en obtain th e m apping into Laplacian e igenspace by com puting th e e igenvectors of th e Laplacian of th e w e igh t m atrix, W . Th e value of th e ith node in th e jth dim ension is given by th e ith elem ent of th e jth e igenvector.
Se gm e ntation in Eige nspace
W e segm ent th e voxels by fitting spline s to th e voxels in e igenspace.
W e begin w ith th e voxels th at are at th e e xtrem itie s of th e structure in e igenspace. W e grow th e spline by adding voxels th at are close to th e grow ing end of th e spline. W e stop grow ing th e spline w h en th e 1D m odel of th e spline doe s not h old, i.e., at th e junctions w h e re th re e or m ore segm ents m e et (neck , groin).
Th e voxels before and afte r segm entation are pre sented in th e figure above.
For each node in each spline , yi, w e h ave th e corre sponding site param ete r, ti, and w e obtain a spline , f(.), such th at yi = f(ti). W e th us obtain th e position of each node along th e ch ain.
For each node w e also h ave xi, th e position in norm al space. W e can use th e (ti,xi) pair to com pute th e sk eleton in norm al space for each segm ent.
3D space to Laplacian Eige nspace : Dim . 1-3, Dim . 4-6 3D space to Laplacian Eige nspace : Dim . 1-3, Dim . 4-6
R e gistration to body parts
W e can registe r th e segm ented body parts to th e k now n body m odel using probabilistic tech niq ue s by m easuring prope rtie s of th e spline s such as length and th ick ne ss.
W e are th us able to re solve am biguitie s lik e in th e e xam ple below.
(c) Sk ele ton Curve(b) Se gm e nte d (e ) Conne cte d sk ele ton(d) Conne ctivity Graph(a) Model
Expe rim e ntal re sults
Th e algorith m w as te sted on diffe rent subjects of varying BMI (body-m ass index). In each case 20 correctly segm ented fram e s w e re selected for e stim ating h um an body m odel param ete rs. Th e re sults are pre sented below.
Conclusion
Transform ation to Laplacian e igenspace is ve ry useful in orde r to pe rform segm entation of connected articulated ch ains and h as advantage s over oth e r m eth ods.
Th e 1D structure of th e voxels in Eigenspace can be e xploited for segm entation using 1D spline s. Distance pre se rving transform ations such as LLE w ill not extract th e 1D structure.
Effect of articulation is rem oved. Th e pose of th e subject doe s not m atte r. W e are also able to obtain th e position of each voxel along th e articulated ch ain.
Subje ct B: 165cm , 64k g Subje ct C: 172cm , 52k g Subje ct D: 178cm , 115k gSubje ct A: 184cm , 72k g
Estim ating h um an body sk e le ton m ode l
Th e h um an body m odel consists of joint locations, bone length s, and param ete rs of th e supe r-q uadrics of various body segm ents.
Th e sk eleton body m odel (joint locations and bone length s) can be com puted by fitting line segm ents to th e sk eletons e stim ated in Figure (e ) above. Optim isation of th e se param ete rs can be pe rform ed using 20 diffe rent fram e s.
Once th e sk eleton m odel param ete rs are initialised and optim ised using th e com puted sk eletons, w e e stim ate th e supe r q uadric param ete rs for th e diffe rent body segm ents using th e voxels.
(a) Afte r Initialisation
Model sk ele ton supe rim pose d w ith com pute d sk ele ton for a fe w fram e s.
(b) Afte r final optim isation
Com puting supe r-q uadric param e te rs
Supe r q uadric param ete rs for each body segm ent are com puted by conside ring th e radial profile of th at body segm ent for all th e fram e s. Once th e pose and sk eleton param ete rs are obtained, th e voxels belonging to each body segm ent can be segm ented and th e ir radial profile along th e ir principal axis plotted. Th e radial profile is th e th ick ne ss of th e segm ent along th e principal axis. Th e radial profile for som e segm ents are plotted below.
Supe r-q uadric Radial Profile Vie w
Com pute d radial profile s for diffe re nt se gm e nts
(a) Trunk (b) H e ad (c) Fore arm (d) Le g
Sam ple im age s from m ultiple cam e ras w ith back ground subtraction
Voxels in 3D space
![· PDF fileghansha das richpal singh shiv bux singh ... suresh kumar lakhan singh chena ram ... narendra kumar kashi ram puna ram bra.] lal mort ram](https://static.fdocument.pub/doc/165x107/5a94ebf77f8b9a8b5d8cc3a1/das-richpal-singh-shiv-bux-singh-suresh-kumar-lakhan-singh-chena-ram-narendra.jpg)
