USRE38420EExpiredUtility

Derivation of studio camera position and motion from the camera image

Assignee: BRITISH BROADCASTING CORPPriority: Aug 12, 1992Filed: Aug 12, 1993Granted: Feb 10, 2004

Est. expiryAug 12, 2012(expired)· nominal 20-yr term from priority

Inventors:Graham Thomas

G06V 10/147G06T 7/20H04N 19/54H04N 5/144H04N 19/53H04N 19/527G06T 2207/30244H04N 5/2224G06T 7/70

PatentIndex Score

Cited by

References

100

Claims

Abstract

Studio camera position and motion may be derived from the camera image by separating out the background and deriving from a background having a number of areas of hue and/or brightness different from adjacent areas estimates of movement from one image to the next. The initial image is used as a reference and amended with predicted motion value. The amended image is compared with incoming images and the result used to derive translation and scale change information. Once the proportion of the reference image contained in an incoming image falls below a threshold a fresh reference image is adopted.

Claims

exact text as granted — not AI-modified

I claim:

1. A method of measuring the translation and scale change in a sequence of video images, the method comprising:
storing a first image in said sequence to form a stored image;
forming a prediction of the translation and scale change from said stored image to a further image in said sequence;
comparing said stored image to said further image by transforming at least one of said stored image byand said further image based on said prediction of the translation and scale change; to form a transformed first image;
comparing a further image in said sequence with said transformed first image;
deriving measurements of translation and scale change between said stored image and said further image from said comparison and said prediction; and
replacing said first image with a new incoming image when the image area common to both said first image and incoming image falls below a given proportion of the whole image area of said incoming image.

2. The method of claim 1 , wherein only the background areas of said video images are used in the measurement of translation and scale change.

3. The method of claim 2 , wherein a signal is used to separate foreground and background and said signal is derived using chroma-key techniques.

4. The method of claim 2 , wherein a signal is used to separate foreground and background portions of the images and said signal is derived using motion detection methods to identify objects moving with a different motion from that predicted for the background.

5. The method of claim 1 , wherein:
each image is a single-component signal derived from a camera viewing a scene containing a background of near-uniform color;
said background is divided into a plurality of areas each having a hue and/or brightness different from the hue and/or brightness of adjacent areas to allow the generation of a key signal by chroma-key techniques; and
wherein said single-component signal is formed from a three-component camera signal so to accentuate differences in hue and/or brightness of individual areas of said background to enable motion estimation.

6. A method of measuring the translation and scale change in a sequence of video images of a camera signal derived by a camera, comprising:
storing a first image of said sequence to form a stored image said image comprising a single component signal derived by a camera viewing a scene containing a background of near-uniform color, and said background being divided into a plurality of areas each having a hue and/or brightness different from the hue and/or brightness of adjacent areas to allow generation of a key signal by chroma-key techniques;
forming said single-component signal from said camera signal so as to accentuate differences in hue and/or brightness of individual areas of the background to enable motion estimation;
forming a prediction of the translation and scale change from said stored image to a further image in said sequence;
comparing said stored image to said further image by transforming at least one of said stored image byand said further image based on said prediction of the translation and scale change; to form a transformed first image;
comparing a further image in said sequence with said transformed first image; and
deriving measurements of translation and scale change between said stored image and said further image from said comparison and said prediction.

7. The method of claim 6 , wherein said background is divided into a plurality of areas, each area having one of two hues and/or brightnesses.

8. The method of claim 6 , wherein said areas of the background are square.

9. The method of claim 1 or 6 , wherein said translation and scale change are predicted by the computation of a number of simultaneous equations each of which relate said translation and scale change to spatial and temporal gradients at a point in the reference image and which are solved to yield a least-squares solution for the motion parameters.

10. The method of claim 1 or 6 , comprising:
selecting a number of measurement points in said first image for motion estimation; and
replacing said first image with a new incoming image when the number of measurement points which lie in areas of background visible in both said first image and a given incoming image falls below a given proportion of the total number of measurement points.

11. The method of claim 1 or 6 , comprising replacing said first image if the scale change between an incoming image and the reference image exceeds a given factor.

12. The method of claim 1 or 6 , comprising spatially prefiltering said first image prior to storage and comparison.

13. The method of claim 10 , wherein said measurement points lie in a regular array in the image.

14. The method of claim 10 wherein said measurement points are chosen to lie at points of high spatial gradient.

15. The method of claim 1 or 6 , comprising storing replaced reference images for later use.

16. Apparatus for measuring the translation and scale change in a sequence of video images, comprising:
means for acquiring said sequence of images;
storage means for storing a first image in said sequence to form a stored image;
means for forming a prediction of the translation and scale change from said stored image to a further image in said sequence;
means for comparing said stored image to said further image by transforming at least one of said stored image by and said further image based on said prediction of the translation and scale change to form a transformed first image;
means for comparing said transformed first image with a further image in the sequence ;
means for deriving measurements of translation and scale change between said stored image and said further image from said comparison and said prediction; and
means for replacing said first image with a new incoming image when the image area common to both said first image and incoming image falls below a given proportion of the whole image area of said incoming images.

17. The apparatus of claim 16 , wherein said means for deriving operates only on the background areas of the images, said means for deriving comprising means for separating foreground and background portions of the images using motion techniques to identify objects moving with a different motion from that predicted for the background.

18. The apparatus of claim 16 , wherein said means for deriving operates only on the background areas of the images, and said means for deriving comprising a key generator for generating a chroma-key to separate foreground and background.

19. The apparatus of claim 16 , comprising:
means for generating a single-component signal from a camera viewing a scene containing a background of near-uniform color, said background being divided into a plurality of areas each having a hue and/or brightness different from the hue and/or brightness of adjacent areas to allow the generation of a key signal by chroma-key techniques, and wherein the single-component signal is formed from a three-component camera signal so as to accentuate differences in hue and/or brightness of individual areas of the background to enable motion estimation.

20. Apparatus for measuring the translation and scale change in a sequence of video images derived by a camera, comprising:
means for storing a first image of said sequence to form a stored image, said image comprising a single component signal derived by a camera viewing a scene containing a background of near-uniform color, said background being divided into a plurality of areas each having a hue and/or brightness different from the hue and/or brightness of adjacent areas to allow generation of a key signal by chroma-key techniques;
means for forming said single-component signal from said camera signal so as to accentuate differences in hue and/or brightness of individual areas of the background to enable motion estimation;
means for forming a prediction of the translation and scale change from said stored image to a further image in said sequence;
means for comparing said stored image to said further image by transforming at least one of the stored image by a and said further image based on said prediction of the translation and scale change to form a transformed first image;
means for comparing a further image in said sequence with said transformed first image; and
means for deriving, from said comparison and said prediction, measurements of translation of scale change between said stored image and said further image.

21. The apparatus of claim 19 or 20 , wherein said background is divided into a plurality of areas each area having one or two hues and/or brightnesses.

22. The apparatus of claim 19 or 20 , wherein said areas of the background are square.

23. The apparatus of claim 19 or 20 , comprising:
means for predicting the translation and scale change by the computation of a number of simultaneous equations each of which relate the translation and scale change to spatial and temporal gradients at a point in the first image and which are solved to yield a least-squares solution for the motion parameters.

24. The apparatus of claim 19 or 20 , wherein the replacing means comprises:
means for selecting a number of measurement points in the first image for motion estimation and for replacing the first image with a new incoming image when the number of measurement points which lie in areas of background visible in both the first image and a given incoming image falls below a given proportion of the total number of measurement points.

25. The apparatus of claim 16 or 20 , comprising a spatial filter for filtering the images prior to storage and comparison.

26. The apparatus of claim 16 or 20 , comprising a further storage means for storing a replaced reference image for future use.

27. The method of claim 1 wherein said step of comparing said stored image to said further image comprises substeps of:

transforming said stored image by said prediction of the translation and scale change to form a transformed first image; and

comparing a further image in said sequence with said transformed first image.

28. The method of claim 6 wherein said step of comparing said stored image to said further image comprises substeps of:

transforming said stored image by said prediction of the translation and scale change to form a transformed first image; and

comparing a further image in said sequence with said transformed first image.

29. The apparatus of claim 16 wherein said means for comparing said stored image to said further image comprises:

means for transforming said stored image by said prediction of the translation and scale change to form a transformed first image; and

means for comparing a further image in said sequence with said transformed first image.

30. The apparatus of claim 20 wherein said means for comparing said stored image to said further image comprises:

means for transforming said stored image by said prediction of the translation and scale change to form a transformed first image; and

means for comparing a further image in said sequence with said transformed first image.

31. Apparatus for measuring the translation and scale change in a sequence of video images, comprising:

an image storage unit that stores a first image in said sequence to form a stored image;

a prediction unit that forms a prediction of the translation and scale change from said stored image to a further image in said sequence;

an image transformer that compares said stored image to said further image by transforming at least one of said stored image and said further image based on said prediction of the translation and scale change;

a measurement unit that derives measurements of translation and scale change between said stored image and said further image from said comparison and said prediction; and

a refresh signal generator that replaces said first image with a new incoming image when the image area common to both said first image and incoming image falls below a given proportion of the whole image area of said incoming images.

32. The apparatus of claim 31 , wherein said measurement unit operates only on the background areas of the images, said measurement unit separating foreground and background portions of the images using motion techniques to identify objects moving with a different motion from that predicted for the background.

33. The apparatus of claim 31 , wherein said measurement unit operates only on the background areas of the images, and said measurement unit comprises a key generator for generating a chroma- key to separate foreground and background.

34. The apparatus of claim 31 , comprising:
means for generating a single - component signal from a camera viewing a scene containing a background of near - uniform color, said background being divided into a plurality of areas each having a hue and/or brightness different from the hue and/or brightness of adjacent areas to allow the generation of a key signal by chroma - key techniques, and wherein the single - component signal is formed from a three - component camera signal so as to accentuate differences in hue and/or brightness of individual areas of the background to enable motion estimation.

35. Apparatus for measuring the translation and scale change in a sequence of video images derived by a camera, comprising:
an image storage unit that stores a first image of said sequence to form a stored image, said image comprising a single component signal derived by a camera viewing a scene containing a background of near - uniform color, said background being divided into a plurality of areas each having a hue and/or brightness different from the hue and/or brightness of adjacent areas to allow generation of a key signal by chroma - key techniques;
a signal former that forms said single - component signal from said camera signal so as to accentuate differences in hue and/or brightness of individual areas of the background to enable motion estimation;

a prediction unit that forms a prediction of the translation and scale change from said stored image to a further image in said sequence;

an image transformer that compares said stored image to said further image by transforming at least one of the stored image and said further image based on said prediction of the translation and scale change; and

a measurement unit that derives, from said comparison and said prediction, measurements of translation of scale change between said stored image and said further image.

36. The apparatus of claim 35 wherein said background is divided into a plurality of areas each area having one or two hues and/or brightnesses.

37. The apparatus of claim 35 , wherein said areas of the background are square.

38. A method of deriving a measure of translation and scale change from a camera output signal, the method comprising:

providing a background with a pattern comprising a plurality of areas of a first tone and a plurality of areas of a second tone arranged so that transitions between areas occur along at least two axes, whereby the transitions provide reference points from which measures of both translation and scale change may be determined;

receiving said camera output signal defining a camera image from a camera viewing a scene containing at least a portion of said background;

detecting said first and second tones in the camera output signal by chroma - keying; and

deriving from results of said detecting step a measure of translation and scale change.

39. A method according to claim 38 , wherein the first and second tones differ in hue.

40. A method according to claim 38 , wherein the first and second tones differ in brightness.

41. A method according to claim 38 , wherein the first and second tones are of substantially equal brightness.

42. A method according to claim 38 , wherein the pattern comprises a chequered pattern.

43. A method according to claim 38 , wherein said chroma- keying is used to remove foreground objects from the image prior to determination of translation and scale change.

44. A method according to claim 38 wherein the camera output signal is spatially filtered prior to storage and comparison.

45. A method according to claim 38 , wherein said measures of translation and scale change are determined using a stored reference image.

46. A method according to claim 45 , wherein translation and scale change are determined by transforming the reference image and comparing with the camera image.

47. A method according to claim 46 , wherein said transforming is based on a prediction of the translation and scale change.

48. A method according to claim 45 , wherein translation and scale change are determined by computation of a number of simultaneous equations each of which relate the translation and scale change to spatial and temporal gradients at a point in the reference image and which are solved to yield a least- squares solution for the motion parameters.

49. A method according to claim 45 , wherein the stored reference image is replaced when the accumulated translation and scale change exceeds a given threshold.

50. A method according to claim 38 , wherein translation and scale change are determined from a sub- set of the pixels in the camera image.

51. A method according to claim 50 , wherein the sub- set of pixels lie in a regular array in the camera image.

52. A method according to claim 50 , wherein the sub- set of pixels are chosen to lie at points of high spatial gradient.

53. A method according to claim 38 , wherein a stored reference image is compared to the camera image after transforming the reference image based on a prediction of translation and scale change and a measure of translation and scale change derived from the comparison.

54. A method according to claim 53 , wherein the reference image comprises an earlier camera image.

55. A method according to claim 54 , wherein the reference image is replaced when the overlap between the reference image and the current camera image falls below a given proportion.

56. A method according to claim 38 , wherein data obtained from a sensor on the camera is used in derivation of said measures of translation and scale change.

57. Apparatus for deriving at least one parameter representing the movement of a camera from a camera image, said camera viewing at least a portion of a background having a pattern comprising a plurality of areas of a first tone and a plurality of areas of a second tone arranged so that transitions between areas occur along at least two axes, whereby the transitions provide reference points from which measures of both translation and scale change may be determined, the apparatus comprising:

means for receiving an output signal from said camera, said signal defining a camera image;

means for detecting said first and second tones in the camera image by chroma - keying; and

means for deriving from results of said detecting means a measure of translation and scale change.

58. Apparatus according to claim 57 further comprising said background.

59. Apparatus according to claim 57 further comprising said camera.

60. Apparatus according to claim 57 further comprising said background and said camera.

61. Apparatus according to claim 57 , wherein the first and second tones differ in hue.

62. Apparatus according to claim 57 , wherein the first and second tones differ in brightness.

63. Apparatus according to claim 57 , wherein the first and second tones are of substantially equal brightness.

64. Apparatus according to claim 57 , wherein the pattern comprises a chequered pattern.

65. Apparatus according to claim 57 , wherein said detecting means is arranged to remove foreground objects from the image prior to determination of translation and scale change.

66. Apparatus according to claim 57 comprising means for spatially filtering the camera image prior to storage and comparison.

67. Apparatus according to claims 57 , further comprising an image store that stores a reference image for use by said deriving means.

68. Apparatus according to claim 57 , wherein the deriving means is configured to determine translation and scale change by transforming the reference image and comparing with the camera image.

69. Apparatus according to claim 68 , wherein said transforming is based on a prediction of the translation and scale change.

70. Apparatus according to claim 67 , wherein translation and scale change are determined by computation of a number of simultaneous equations each of which relate the translation and scale change to spatial and temporal gradients at a point in the reference image and which are solved to yield a least- squares solution for the motion parameters.

71. Apparatus according to claims 67 wherein the stored reference image is replaced when the accumulated translation and scale change exceeds a given threshold.

72. Apparatus according to claim 57 , wherein said deriving means is configured to determine translation and scale change from a sub- set of the pixels in the camera image.

73. Apparatus according to claim 72 , wherein the sub- set of pixels lie in a regular array in the camera image.

74. Apparatus according to claim 72 wherein the sub- set of pixels are chosen to lie at points of high spatial gradient.

75. Apparatus according to claim 57 , further comprising a sensor on the camera that provides data for use in derivation of said measures of translation and scale change.

76. Apparatus according to claim 57 , wherein said deriving means comprises digital signal processing apparatus.

77. Apparatus for deriving at least one parameter representing the movement of a camera from a camera image, said camera viewing at least a portion of a background having a pattern comprising a plurality of areas of a first tone and a plurality of areas of a second tone arranged so that transitions between areas occur along at least two axes, whereby the transitions provide reference points from which measures of both translation and scale change may be determined, the apparatus comprising:

an input receiving an output signal from said camera, said signal defining a camera image; and

a signal processing system that 1 ) detects said first and second tones in the camera image by chroma - keying; and 2 ) derives from the results of said detection a measure of translation and scale change.

78. Apparatus according to claim 77 further comprising said background.

79. Apparatus according to claim 77 further comprising said camera.

80. Apparatus according to claim 77 further comprising said background and said camera.

81. Apparatus according to claim 77 , wherein the first and second tones differ in hue.

82. Apparatus according to claim 77 , wherein the first and second tones differ in brightness.

83. Apparatus according to claim 77 , wherein the first and second tones are of substantially equal brightness.

84. Apparatus according to claim 77 , wherein the pattern comprises a chequered pattern.

85. Apparatus according to claim 77 , wherein said signal processing system removes foreground objects from the image prior to determination of translation and scale change.

86. Apparatus according to claim 77 , wherein said signal processing system determines translation and scale change by transforming the reference image and comparing with the camera image.

87. Apparatus according to claim 77 , further comprising an image store that stores a reference image for use by said signal processing system.

88. Apparatus according to claim 87 comprising a filter that spatially filters the camera image prior to storage and comparison.

89. Apparatus according to claim 86 , wherein said transforming is based on a prediction of the translation and scale change.

90. Apparatus according to claim 87 , wherein translation and scale change are determined by computation of a number of simultaneous equations each of which relate the translation and scale change to spatial and temporal gradients at a point in the reference image and which are solved to yield a least- squares solution for the motion parameters.

91. Apparatus according to claim 87 , arranged to replace the stored reference image when the accumulated translation and scale change exceeds a given threshold.

92. Apparatus according to claim 77 , wherein said signal processing system determines translation and scale change from a sub- set of the pixels in the camera image.

93. Apparatus according to claim 92 , wherein the sub- set of pixels lie in a regular array in the camera image.

94. Apparatus according to claim 92 wherein the sub- set of pixels are chosen to lie at points of high spatial gradient.

95. Apparatus according to claim 77 , further comprising a sensor on the camera that provides data for use in derivation of said measures of translation and scale change.

96. Apparatus according to claim 77 , wherein said deriving means comprises digital signal processing apparatus.

97. The apparatus of claim 35 wherein said areas of the background are quadrilaterals.

98. The method of claim 38 wherein said areas of the first tone and areas of the second tone are quadrilaterals.

99. The apparatus of claim 57 wherein said areas of the first tone and areas of the second tone are quadrilaterals.

100. The apparatus of claim 77 wherein said areas of the first tone and areas of the second tone are quadrilaterals.

Join the waitlist — get patent alerts

Track USRE38420E — get alerts on status changes and closely related new filings.

We store only your email — no account needed. See our privacy policy.