Mobile holographic simulator of bowling pins and virtual objects

Abstract

A system with automatic positioning controls for the holographic display of three-dimensional objects interacting in real time with objects of the real world. The person observing the screen ( 18 ) has the sensation that the objects are real, which is achieved thanks to a real time perspective correction system ( 1 ). The system is used in particular to simulate bowling pins ( 9 ). It is positioned on the lane required by means of a dolly ( 6 ). The system determines a relation between the information received from the sensors: video cameras ( 1 ) ( 3 ), dimensions ( 26 ), weight ( 14 ) ( 27 ) and the mathematical simulation models. When the ball is bowled, its physical features are measured and when it hits the pin deck, some holographic pins are displayed on the screen ( 19 ). The screen holographically arranges the projected objects over what the bowler ( 18 ) sees behind the screen ( 19 ). The simulated movements of the pins and the physical interactions with the real objects are determined by exploiting the theory of mechanics in three-dimensional space.

1 - 43 . (canceled) 
   
   
       44 . System that offers the visual and sonorous perception of virtual objects interacting dynamically and in real time with real objects having known physical dimensions, characterised by a physically and geometrically known surface on which the real objects may move, featuring a transparent holographic or almost transparent screen set between the real objects and the bowler with the function of showing the bowler virtual perspective objects interacting in real time with real objects and simultaneously allowing the bowler to see the real objects situated on the part of surface that he sees through the transparent holographic screen so that he sees the virtual objects as real, characterised by a projection system that projects a sequence of images showing virtual objects on the holographic screen dynamically interacting with real objects, characterised by a system that detects the position of the real objects, characterised by a system that re-creates sound effects that simulate the sounds produced when the virtual objects collide with the real objects and when the virtual objects collide with other virtual objects in real time, characterised by a control unit that processes the signals of the sensors working in real time and using a physical mathematical model that reproduces the real situation with the aim to generate and send the sequence of images from the projector to the screen and to generate and send the signals to the system to create the sound effects. 
   
   
       45 . System that offers the visual and sonorous perception of virtual objects interacting dynamically and in real time with real objects having known physical dimensions, characterised by a physically and geometrically known surface on which the real objects may move, featuring a transparent holographic or almost transparent screen set between the real objects and the bowler with the function of showing the bowler virtual perspective objects interacting in real time with real objects and simultaneously allowing the bowler to see the real objects situated on the part of surface that he sees through the transparent holographic screen so that he sees the virtual objects as real, characterised by a projection system that projects a sequence of images showing virtual objects on the holographic screen dynamically interacting with real objects, characterised by a system that detects the position of the real objects, characterised by a system that re-creates sound effects that simulate the sounds produced when the virtual objects collide with the real objects and when the virtual objects collide with other virtual objects in real time, characterised by a control unit that processes the signals of the sensors working in real time and using a physical mathematical model that reproduces the real situation with the aim to generate and send the sequence of images from the projector to the screen and to generate and send the signals to the system to create the sound effects, characterised by a system that detects the position in the space of the bowler with the aim to use this point as the perspective point in generating the images to be projected on the holographic screen to provide a realistic perspective view on the screen. 
   
   
       46 . System that offers the visual and sonorous perception of virtual objects interacting dynamically and in real time with real objects having known physical dimensions, characterised by a physically and geometrically known surface on which the real objects may move, featuring a transparent holographic or almost transparent screen set between the real objects and the bowler with the function of showing the bowler virtual perspective objects interacting in real time with real objects and simultaneously allowing the bowler to see the real objects situated on the part of surface that he sees through the transparent holographic screen so that he sees the virtual objects as real, characterised by a projection system that projects a sequence of images showing virtual objects on the holographic screen dynamically interacting with real objects, characterised by a system that detects the position of the real objects, characterised by a system that re-creates sound effects that simulate the sounds produced when the virtual objects collide with the real objects and when the virtual objects collide with other virtual objects in real time, characterised by a control unit that processes the signals of the sensors working in real time and using a physical mathematical model that reproduces the real situation with the aim to generate and send the sequence of images from the projector to the screen and to generate and send the signals to the system to create the sound effects, characterised by a system that determines the type of object among all those of known physical dimensions, characterised by a system that detects the angular speeds and that transmits them to the control unit that uses them to increase the realism of the dynamic simulation of the virtual objects. 
   
   
       47 . System that offers the visual and sonorous perception of virtual objects interacting dynamically and in real time with real objects having known physical dimensions, characterised by a physically and geometrically known surface on which the real objects may move, featuring a transparent holographic or almost transparent screen set between the real objects and the bowler with the function of showing the bowler virtual perspective objects interacting in real time with real objects and simultaneously allowing the bowler to see the real objects situated on the part of surface that he sees through the transparent holographic screen so that he sees the virtual objects as real, characterised by a projection system that projects a sequence of images showing virtual objects on the holographic screen dynamically interacting with real objects, characterised by a system that detects the position of the real objects, characterised by a system that re-creates sound effects that simulate the sounds produced when the virtual objects collide with the real objects and when the virtual objects collide with other virtual objects in real time, characterised by a control unit that processes the signals of the sensors working in real time and using a physical mathematical model that reproduces the real situation with the aim to generate and send the sequence of images from the projector to the screen and to generate and send the signals to the system to create the sound effects, characterised by a system that detects the position in the space of the bowler with the aim to use this point as the perspective point in generating the images to be projected on the holographic screen to provide a realistic perspective view on the screen, characterised by a system that determines the type of object among all those of known physical dimensions, characterised by a system that detects the angular speeds and that transmits them to the control unit that uses them to increase the realism of the dynamic simulation of the virtual objects. 
   
   
       48 . Arrangement in accordance with patent  claim 44 ,  45 ,  46 ,  47  characterized by a holographic screen that is positioned between the bowler ( 18 ) and the position where the virtual objects should be if they were real ( 9 ); the screen is able to refract the high intensity rays emitted by the projection system to obtain visible virtual objects and characterized by the fact that it lets through any light of minor intensity without alteration or almost without alteration, being the light typically reflected by the real objects, showing a transparent view of the real objects just as they are. 
   
   
       49 . The system of  claim 48  includes air screens, fog screens with transparent liquids or gas, holographic crystals screens, made of almost transparent material, screens made of almost transparent fabric or of flexible transparent rubber. 
   
   
       50 . In the system of  claim 48  the screen is positioned in the point with the smallest perspective error from the point of view of the bowler and in that nearest the position in the space of the first virtual object ( 9 ). The screen must not stop the movement of the real objects on the surface where they move. 
   
   
       51 . The system of  claim 48  includes all types of solid and rigid screens that cannot be crossed by real objects, so the screen is raised off the known movement surface by at least the distance necessary to let the real objects through. 
   
   
       52 . The system of  claim 48  includes all types of non-solid or non-rigid screens that can be crossed by real objects, which are positioned on the known movement surface in the real point associated unmistakeably with the point of the mathematical model in which the virtual object nearest the bowler is positioned. 
   
   
       53 . Arrangement in accordance with patent  claim 44 ,  45 ,  46 ,  47  with the function of determining the position of the real objects; the system consists of a camera or an optic system. 
   
   
       54 . Arrangement in accordance with patent  claim 53  made up of a camera of which the images are processed to analyse the chromatic variations to determine the position of the objects on the known surface. This is determined using the mathematical model of the known surface, the model of the real objects and the physical/geometric/optical model of the camera that sets the images in unmistakeable relation with the real system in order to deduce the speed and the trajectory of the real objects in real time. 
   
   
       55 . Arrangement in accordance with patent  claim 53  made up of a system of photocells that detect the trajectory and the speed of the real object at a known time in a known point, made up of photocells crossed with known position and direction with the aim to detect the crossing times and to calculate the direction and the linear speed of the objects that cross it based on the times. 
   
   
       56 . Arrangement in accordance with patent  claim 46 , 47  with the function of determining the type of real object in order to extract and use the physical data that are saved and that are unmistakeable for each real object of the known set of objects. The system includes a camera or a radio identification system. 
   
   
       57 . Arrangement in accordance with patent  claim 56  made up of a camera of which the images are analysed in the chromatic variations and based on the comparison of probabilistic methods and chromatic filters and using the mathematical model of the system and of the camera to define the shape, size and superficial chromatic distribution of the object. 
   
   
       58 . The information stated in  claim 57  is used as an unmistakeable access key to extract the physical dimensions of the object from a limited archive. 
   
   
       59 . Arrangement in patent  claim 57  with the aim to determine the angular speed of the real objects by analysing the variations in the superficial chromatic distribution of the real object, since their geometric shape and position in the space in relation to the camera are known. 
   
   
       60 . Arrangement in accordance with patent  claim 56  with the function of determining the type of real object with the aim to extract and use the physical information, known unmistakeably, for each real object and being part of a known set of objects. The system consists of a receiver in radio frequency and of an rfid tag with unmistakeable code applied to the real objects. Each rfid is an unmistakeable access key to an archive of the physical/geometric dimensions of the object to be used to generate the simulation. 
   
   
       61 . Arrangement in accordance with patent  claim 60  characterized in that there is a coding system that enables each real object to send a code in radio frequency that represents its information in terms of mass, dimensions, type of material, balances, moment of inertia, colour, friction and number of holes. 
   
   
       62 . Arrangement in accordance with patent  claim 56  made up of a gyroscopic sensor situated in the real object to determine the angular speed and to send this information in radio frequency to a receiver that sends it to the control unit. 
   
   
       63 . Arrangement in accordance with patent  claim 45 ,  47  with the function of detecting the position of the bowler in the space with the aim to use this point as a prospective point in generating the sequence of images projected on the screen and to thus obtain a projection that the bowler perceives based on his position in the space. A camera is used to obtain an unmistakeable relationship between the chromatic variations of the camera and the position in the space of the eyes of the bowler. This relationship is obtained via probabilistic comparison methods of the chromatic variations with the typical chromatic variations of human shapes, the knowledge of the surface on which the bowler is standing and using the mathematical model of the camera of which the position and direction in the space are known. 
   
   
       64 . Arrangement in accordance with patent  claim 44 ,  45 ,  46 ,  47  including a projector used to project the images on the holographic screen. 
   
   
       65 . Arrangement in accordance with patent  claim 44 ,  45 ,  46 ,  47  including two projectors used to project the images on the holographic screens that can be crossed by the real objects and with the aim to reduce the shadows generated by the real objects as they move. 
   
   
       66 . The projectors in patent  claim 65  display the sequences of images turned, reflected and distorted to be adapted to where the projectors are positioned so that all the projectors display the same image at the same time on the screen to obtain one single projection on the screen; therefore where the light of one projector is obscured by an object, the other projector completes the image in the shadowed zone. 
   
   
       67 . The projectors in patent  claim 65  are positioned so that if an object obscures the projector light and creates a shadow on the screen, then the second projector reaches the zone of screen in the shadow with its rays. 
   
   
       68 . In accordance with patent  claim 67  the projectors are positioned behind the screen, one in the right side and one in the left side and they project on the screen in the bowler's direction. 
   
   
       69 . In accordance with patent  claim 67  the projectors are positioned, one in front of the screen and one behind it, so that one projects towards the bowler and one projects in the opposite direction. 
   
   
       70 . Arrangement in accordance with patent  claim 44 ,  45 ,  46 ,  47  characterized by the fact that the control unit ( 8 ) receives the data sent from the sensors and produces a physical simulation of the virtual objects, as near as possible to reality, using a mechanical three-dimensional model in real time. 
   
   
       71 . Arrangement in accordance with patent  claims 44 ,  45 ,  46 ,  47  characterized in that with this screen ( 19 ) we can change the properties of the real objects, such as the colour, projecting a virtual object over the real object in real time. 
   
   
       72 . Arrangement in accordance with patent  claim 44 ,  45 ,  46 ,  47  characterized in that the system acquires ( 8 ) and saves the physical parameters and is used to collect useful statistics to improve the bowler's ability; these parameters are transmitted to the control unit. 
   
   
       73 . Arrangement in accordance with patent  claim 44 ,  45 ,  46 ,  47  characterized in that we can see a replay of the real and virtual objects. 
   
   
       74 . Arrangement in accordance with patent  claim 44 ,  45 ,  46 ,  47  characterized in that we can modify some of the parameters of the real objects to simplify the game at pleasure. 
   
   
       75 . A method for creating an optical and sonorous illusion towards a bowler standing in a limited space and far enough away from the holographic screen to perceive the perspective simulation as credible, to see the virtual objects interacting with the real objects, tracing the movements of known real objects in real time on a known surface, setting a transparent screen between the bowler and the space where the objects move, on which the sequences of images are projected that show virtual objects that simulate the physical, dynamic, optical and spatial behaviour of real objects in real time while the real objects are seen through the transparent screen. The sequence of images generated are generated so that the bowler perceives the holographic objects almost as if there were actually real objects behind the screen interacting dynamically and in real time with the real objects.