Abstract: In current interactive television
schemes, the viewpoints should be manipulated by the user. However,
there is no efficient method, to assist a user in automatically
identifying and tracking the optimum viewpoint when the user
observes the object of interest because many objects, most often
humans, move rapidly and frequently. This paper proposes a novel
framework for determining and tracking the virtual camera to best
capture the front of the person of interest (PoI). First, one PoI is
interactively chosen in a segmented 3D scene reconstructed by space
carving method. Second, key points of the human torso of the PoI are
detected by using a model-based method and the human's global motion
including rotation and translation is estimated by using a
close-formed method with 3 corresponding points. At the last step,
the front direction of PoI is tracked temporally by using the
unscented particle filter (UPF). Experimental results show that the
method can properly compute the front direction of the PoI and
robustly track the best viewpoints.
TONG Ming-lei (仝明磊)
. Viewpoint Manipulation for Interactive Television by Using Human Pose
Estimation[J]. Journal of Shanghai Jiaotong University(Science), 2011
, 16(5)
: 538
-542
.
DOI: 10.1007/s12204-011-1184-6
1 Tanimoto M. Overview of free view point television [J].
Signal Processing: Image Communication, 2006, 21(6): 454-461.
2 Aljoscha S, Karsten M, Philipp M, et al. 3D video and free
view point video technologies, applications and MPEG standards [C]//
Proceedings of the 2006 IEEE International Conference on
Multimedia and Expo. Toronto, Ontario, Canada：IEEE Computer
Society, 2006: 2161-2164.
3 Toshiaki F, Kensakum, Kazuya T, et al. Multipoint measuring
system for video and sound: 100-camera and microphone system [C]//
Proceedings of the 2006 IEEE International Conference on
Multimedia and Expo. Toronto, Ontario, Canada：IEEE Computer
Society, 2006: 437-440.
4 Takeo K, Peter R, Narayanan P J. Virtualized reality:
Constructing time varying virtual worlds from real world events [J].
IEEE Multimedia, 1997, 4(1): 34-47.
5 Saied M, Tai L C, Philippe G. Virtual view generation for 3D
digital video [J]. IEEE Multimedia, 1997, 4(1): 18-26.
6 Matusik W, Chris B, Ramesh R, et al. Image based visual hulls [C]// In Proceedings of the 27th Annual Conference on Computer Graphics and Interactive Techniques. New Orleans, Louisiana, USA: ACM Press, 2000: 369-374.
7 Takashi M, Xiaojun W, Takeshi T, et al. Real-time dynamic 3D
object shape reconstruction and high-fidelity texture mapping for
3D video [J]. IEEE Transaction on Circuits System and Video
Technology, 2004, 14(3): 357-369.
8 Julier S J, Uhlmann J K. A new extension of the Kalman filter
to nonlinear systems [C]// Proceedings of the 11th Annual
International Symposium on Aerospace/Defense Sensing, Simulation,
and Controls. Orlando, Florida, USA: SPIE, 1997: 182-193.
9 Rudolph M, Arnaud D, Nando F. The unscented particle filter
[C]// Advances in Neural Information Processing Systems 13.
Denver, Colorado, USA: MIT Press, 2000: 584-590.
10 Shuhan S, Minglei T, Yuncai L. Model based human motion
tracking using probability evolutionary algorithm [J]. Pattern
Recognition Letters, 2008, 29(13): 1877-1886.
11 Ivana M, Mohan T, Edward H, et al. Human body model
acquisition and tracking using voxel data [J]. International
Journal of Computer Vision. 2003, 53(3): 199-223.
