Endoscope system, controller, control method, and recording medium

Abstract

An endoscope system includes an endoscope, a moving device that moves the endoscope, a storage unit, and a processor. The storage unit stores first position information and first rotation angle information, which defines a rotation angle of an endoscope image of a first region, on the first region in a subject and second position information and second rotation angle information, which defines a rotation angle of an endoscope image of a second region, on the second region in the subject. The processor calculates third rotation angle information on a third region in the subject based on the first and second position information, the first and second rotation angle information, and third position information on the third region. If the third region includes the current imaging region, the processor rotates the endoscope image based on the third rotation angle information and outputs the rotated endoscope image to the display device.

1 . An endoscope system comprising:
 an endoscope that is inserted into a subject and captures an endoscope image in the subject;   a moving device that holds the endoscope and moves the endoscope;   a storage unit; and   a controller including at least one processor,   wherein the storage unit stores first position information and first rotation angle information on a first region in the subject and second position information and second rotation angle information on a second region different from the first region in the subject, the first rotation angle information defining a rotation angle of the endoscope image of the first region, the second rotation angle information defining a rotation angle of the endoscope image of the second region,   the at least one processor calculates third rotation angle information on a third region in the subject on a basis of the first position information, the first rotation angle information, the second position information, the second rotation angle information, and third position information on the third region, the third region being different from the first and second regions,   the at least one processor rotates the endoscope image on a basis of the third rotation angle information if the third region includes a current imaging region that is currently being imaged by the endoscope, and   the at least one processor outputs the rotated endoscope image to a display device.   
     
     
         2 . The endoscope system according to  claim 1 , wherein the at least one processor rotates the endoscope image by image processing. 
     
     
         3 . The endoscope system according to  claim 1 , wherein the moving device is configured to rotate the endoscope about an optical axis of the endoscope, and
 the at least one processor rotates the endoscope image by controlling the moving device so as to rotate the endoscope about the optical axis.   
     
     
         4 . The endoscope system according to  claim 1 , wherein the at least one processor is operable in a manual mode that permits a user to operate the endoscope, and
 in the manual mode, the at least one processor determines the first position information, the first rotation angle information, the second position information, and the second rotation angle information and stores the first position information, the first rotation angle information, the second position information, and the second rotation angle information in the storage unit.   
     
     
         5 . The endoscope system according to  claim 4 , wherein the at least one processor determines the first position information and the first rotation angle information on a basis of a first specific tissue included in the endoscope image, and
 the at least one processor determines the second position information and the second rotation angle information on a basis of a second specific tissue included in the endoscope image.   
     
     
         6 . The endoscope system according to  claim 5 , wherein the storage unit stores a learned model of machine learning of a correspondence between an image including a specific tissue and a type of the specific tissue,
 the at least one processor recognizes the first specific tissue and the second specific tissue in the endoscope image by using the learned model stored in the storage unit,   the at least one processor determines the first position information on a basis of a position of an imaging region of the endoscope image in which the first specific tissue is recognized, and determines the first rotation angle information on a basis of a rotation angle of the first specific tissue in the endoscope image, and   the at least one processor determines the second position information on a basis of a position of an imaging region of the endoscope image in which the second specific tissue is recognized, and determines the second rotation angle information on a basis of a rotation angle of the second specific tissue in the endoscope image.   
     
     
         7 . The endoscope system according to  claim 4 , wherein the controller further includes a user interface that receives a user instruction,
 the at least one processor determines the first position information on a basis of a position of the imaging region at a time of reception of a first instruction by the user interface, and determines the first rotation angle information on a basis of a rotation angle about an optical axis of the endoscope at a time of reception of the first instruction by the user interface, and   the at least one processor determines the second position information on a basis of a position of the imaging region at a time of reception of a second instruction by the user interface, and determines the second rotation angle information on a basis of a rotation angle about the optical axis of the endoscope at a time of reception of the second instruction by the user interface.   
     
     
         8 . The endoscope system according to  claim 4 , wherein the at least one processor stores a first endoscope image and a second endoscope image in the storage unit, the first endoscope image serving as an endoscope image of the first region, the second endoscope image serving as an endoscope image of the second region. 
     
     
         9 . The endoscope system according to  claim 8 , wherein the at least one processor determines which one of the first region, the second region, and the third region includes the current imaging region on a basis of the first endoscope image and the second endoscope image that are stored in the storage unit. 
     
     
         10 . The endoscope system according to  claim 1 , wherein the first position information and the second position information are stored in advance in the storage unit, the first and second position information being determined on a basis of an examination image in the subject captured before a surgical operation. 
     
     
         11 . The endoscope system according to  claim 1 , wherein the at least one processor rotates the endoscope image on a basis of the first rotation angle information if the first region includes the current imaging region, and
 the at least one processor rotates the endoscope image on a basis of the second rotation angle information if the second region includes the current imaging region.   
     
     
         12 . The endoscope system according to  claim 11 , wherein the processor determines which one of the first region, the second region, and the third region includes the current imaging region on a basis of a position of the imaging region, the first position information, and the second position information. 
     
     
         13 . The endoscope system according to  claim 1 , wherein the endoscope is supported so as to pivot about a first pivot axis at a predetermined pivot point fixed to the subject, the endoscope pivots about the first pivot axis so as to move the imaging region between the first region and the second region, and
 the first position information, the second position information, and the third position information each include a rotation angle of the endoscope about the first pivot axis.   
     
     
         14 . The endoscope system according to  claim 13 , wherein the endoscope is supported so as to pivot about a second pivot axis at the predetermined pivot point, the second pivot axis being orthogonal to the first pivot axis, and
 the first position information, the second position information, and the third position information are each three-dimensional information and further include a rotation angle of the endoscope about the second pivot axis.   
     
     
         15 . The endoscope system according to  claim 13 , wherein the moving device includes at least one joint and at least one angle sensor that detects a rotation angle of the at least one joint, and
 the processor calculates a rotation angle of the endoscope about the first pivot axis on a basis of the rotation angle detected by the at least one angle sensor.   
     
     
         16 . The endoscope system according to  claim 1 , wherein the storage unit stores a database in which a type of a specific tissue and rotation angle information are associated with each other, and
 if the specific tissue is included in the endoscope image of the third region,   the processor reads, from the database, the rotation angle information corresponding to the type of the specific tissue in the endoscope image and   the processor rotates the endoscope image on a basis of the read rotation angle information.   
     
     
         17 . The endoscope system according to  claim 1 , wherein the at least one processor calculates a positional relationship between the third position information and the first and second position information and
 the at least one processor calculates the third rotation angle information on a basis of the positional relationship, the first rotation angle information, and the second rotation angle information.   
     
     
         18 . The endoscope system according to  claim 3 , wherein the at least one processor rotates the endoscope image by image processing if a rotation angle of the endoscope about the optical axis reaches a critical angle of a predetermined rotatable range. 
     
     
         19 . The endoscope system according to  claim 1 , wherein the endoscope is a direct-view endoscope or an oblique endoscope. 
     
     
         20 . The endoscope system according to  claim 1 , wherein the endoscope has a curved portion electrically operated to be bent at a tip portion of the endoscope. 
     
     
         21 . The endoscope system according to  claim 1 , wherein the at least one processor is configured to update the first position information or the first rotation angle information to position information or rotation angle information on the current imaging region if the current imaging region is included in the first region, and
 the at least one processor is configured to update the second position information or the second rotation angle information to the position information or the rotation angle information on the current imaging region if the current imaging region is included in the second region.   
     
     
         22 . The endoscope system according to  claim 4 , wherein in the manual mode, the at least one processor calculates the third position information and the third rotation angle information and stores the third position information and the third rotation angle information in the storage unit. 
     
     
         23 . The endoscope system according to  claim 1 , wherein the at least one processor is operable in an autonomous mode for autonomously moving the endoscope by controlling the moving device, and
 the at least one processor calculates the third position information and the third rotation angle information during the autonomous mode.   
     
     
         24 . A controller configured to control an endoscope image that is captured by an endoscope and is displayed on a display device,
 the controller comprising:   a storage unit; and   at least one processor,   wherein the storage unit stores first position information and first rotation angle information on a first region in a subject and second position information and second rotation angle information on a second region different from the first region in the subject, the first rotation angle information defining a rotation angle of the endoscope image of the first region, the second rotation angle information defining a rotation angle of the endoscope image of the second region,   the at least one processor calculates third rotation angle information on a third region in the subject on a basis of the first position information, the first rotation angle information, the second position information, the second rotation angle information, and third position information on the third region, the third region being different from the first and second regions,   the at least one processor rotates the endoscope image on a basis of the third rotation angle information if the third region includes a current imaging region that is currently being imaged by the endoscope, and   the at least one processor outputs the rotated endoscope image to the display device.   
     
     
         25 . A control method for controlling an endoscope image that is captured by an endoscope and is displayed on a display device, by using first position information and first rotation angle information on a first region in a subject and second position information and second rotation angle information on a second region different from the first region in the subject,
 the first rotation angle information defining a rotation angle of the endoscope image of the first region, the second rotation angle information defining a rotation angle of the endoscope image of the second region,   the control method comprising the steps of:   calculating third rotation angle information on a third region in the subject on a basis of the first position information, the first rotation angle information, the second position information, the second rotation angle information, and third position information on the third region, the third region being different from the first and second regions;   rotating the endoscope image on a basis of the third rotation angle information if the third region includes a current imaging region that is currently being imaged by the endoscope, and   outputting the rotated endoscope image to the display device.   
     
     
         26 . A computer-readable non-transitory recording medium in which a control program for causing a computer to perform the control method according to  claim 25  is stored.