Selective redaction of media content

Abstract

In general, various aspects of the present invention provide methods, apparatuses, systems, computing devices, computing entities, and/or the like for identifying and documenting certain subject matter found in media content, as well as selectively redacting the subject matter found in media content. In accordance with various aspects, a method is provided that comprises: obtaining first metadata for media content, the metadata identifying a context and a portion of content for an individual; identifying, based on the context, a certain subject matter for the media content; determining that a particular item associated with the subject matter is present in the portion of content associated with the individual; generating second metadata to document the particular item present in the portion of content; and using the second metadata to selectively redacting the item from the portion of content associated with the individual upon request to do so with respect to the individual.

The invention claimed is: 
     
       1. A method comprising:
 obtaining, by computing hardware, first metadata for media content comprising a first portion of audio content of a first individual involved in the media content and a second portion of audio content of a second individual involved in the media content, wherein the first metadata identifies (1) a context for the media content, (2) the first portion of audio content is associated with the first individual, and (3) the second portion of audio content is associated with the second individual; 
 receiving a first request for the media content, wherein the first request identifies the first individual; 
 identifying, by the computing hardware and based on the context, subject matter for the media content; 
 for the first portion of audio content: 
 segmenting, by the computing hardware, the first portion of audio content into a first sequence of audio content segments; 
 processing, by the computing hardware, each audio content segment of the first sequence of audio content segments using a multilabel classification machine-learning model to generate a first feature representation comprising a plurality of components for the audio content segment, wherein each component of the plurality of components represents a particular keyword associated with the subject matter and provides a prediction as to whether the particular keyword associated with the subject matter is present in the audio content segment; 
 determining, by the computing hardware and based on the prediction provided for a particular component of the plurality of components generated for a first particular audio content segment of the first sequence of audio content segments, that the particular keyword associated with the subject matter is present in the first particular audio content segment; 
 responsive to determining that the particular keyword associated with the subject matter is present in the first particular audio content segment, generating, by the computing hardware, a first identifier for the first particular audio content segment that identifies the particular keyword associated with the subject matter is present in the first particular audio content segment; 
 generating, by the computing hardware based on the first identifier, a first redacted audio content segment for the first particular audio content segment, wherein the particular keyword associated with the subject matter that is present in the first particular audio content segment is absent from the first redacted audio content segment; and 
 reassembling, by the computing hardware, the first sequence of audio content segments with the first particular audio content segment replaced with the first redacted audio content segment to generate a first portion of redacted audio content for the media content; 
 for the second portion of audio content: 
 segmenting, by the computing hardware, the second portion of audio content into a second sequence of audio content segments; 
 processing, by the computing hardware, each audio content segment of the second sequence of audio content segments using the multilabel classification machine-learning model to generate a second feature representation comprising the plurality of components for the audio content segment, wherein each component of the plurality of components represents the particular keyword associated with the subject matter and provides a prediction as to whether the particular keyword associated with the subject matter is present in the audio content segment; 
 determining, by the computing hardware and based on the prediction provided for a particular component of the plurality of components generated for a second particular audio content segment of the second sequence of audio content segments, that the particular keyword associated with the subject matter is present in the second particular audio content segment; and 
 responsive to determining that the particular keyword associated with the subject matter is present in the second particular audio content segment, generating, by the computing hardware, a second identifier for the second particular audio content segment that identifies the particular keyword associated with the subject matter is present in the second particular audio content segment; 
 generating, by the computing hardware, second metadata for the media content, wherein the second metadata comprises the first identifier and the second identifier; and 
 responsive to the first request, providing, by the computing hardware, the media content comprising the first portion of redacted audio content, wherein playing the media content comprising the first portion of redacted audio content results in the particular keyword associated with the subject matter that is present in the first particular audio content segment not being heard spoken by the first individual and the particular keyword associated with the subject matter that is present in the second particular audio content segment being heard spoken by the second individual. 
 
     
     
       2. The method of  claim 1  further comprising identifying, by the computing hardware and based on the first metadata and the first request identifying the first individual, the first portion of audio content of the media content. 
     
     
       3. The method of  claim 1 , wherein generating the first redacted audio content segment comprises at least one of removing the particular keyword associated with the subject matter from the first particular audio content segment, replacing the particular keyword associated with the subject matter in the first particular audio content segment, or altering the particular keyword associated with the subject matter in the first particular audio content segment. 
     
     
       4. The method of  claim 1 , wherein generating the first redacted audio content segment comprises:
 processing the first particular audio content segment using a rules-based model to generate a redaction action, the redaction action involving at least one of removing the particular keyword associated with the subject matter from the first particular audio content segment, replacing the particular keyword associated with the subject matter in the first particular audio content segment, or altering the particular keyword associated with the subject matter in the first particular audio content segment; and performing the redaction action to generate the first redacted audio content segment. 
 
     
     
       5. The method of  claim 1  further comprising:
 receiving a second request for the media content, wherein the second request identifies the second individual; 
 identifying, by the computing hardware based on the second request identifying the second individual, the second identifier in the second metadata for the media content; 
 segmenting, by the computing hardware, the second portion of audio content into the second sequence of audio content segments; 
 generating, by the computing hardware based on the second identifier, a second redacted audio content segment for the second particular audio content segment, wherein the particular keyword associated with the subject matter that is present in the second particular audio content segment is absent from the second redacted audio content segment; 
 reassembling, by the computing hardware, the second sequence of audio content segments with the second particular audio content segment replaced with the second redacted audio content segment to generate a second portion of redacted audio content for the media content; and 
 responsive to the second request, providing, by the computing hardware, the media content comprising the second portion of redacted audio content, wherein playing the media content comprising the second portion of redacted audio content results in the particular keyword associated with the subject matter that is present in the first particular audio content segment being heard spoken by the first individual and the particular keyword associated with the subject matter that is present in the second particular audio content segment not being heard spoken by the second individual. 
 
     
     
       6. The method of  claim 1 , wherein processing each audio content segment of the first sequence of audio content segments using the multilabel classification machine-learning model to generate the first feature representation comprises:
 processing the audio content segment using a speech-to-text engine to generate a text representation of the audio content segment, the text representation comprising a sequence of words spoken in the audio content segment; and 
 processing the text representation using the multilabel classification machine-learning model to generate the first feature representation. 
 
     
     
       7. A system comprising:
 a non-transitory computer-readable medium storing instructions; and 
 a processing device communicatively coupled to the non-transitory computer-readable medium, 
 wherein, the processing device is configured to execute the instructions and thereby perform operations comprising: 
 obtaining first metadata for media content comprising a first portion of video content associated with a first individual involved in the media content and a second portion of video content associated with a second individual involved in the media content, wherein the first metadata identifies (1) a context for the media content, (2) the first portion of video content is associated with the first individual, and (3) the second portion of video content is associated with the second individual; 
 receiving a first request for the media content, wherein the first request identifies the first individual; 
 identifying, based on the context, subject matter for the media content; for the first portion of video content: 
 segmenting the first portion of video content into a first sequence of video content segments; 
 processing each video content segment of the first sequence of video content segments using a multilabel classification machine-learning model to generate a first feature representation comprising a plurality of components for the video content segment, wherein each component of the plurality of components represents a particular object associated with the subject matter and provides a prediction as to whether the particular object associated with the subject matter is present in the video content segment; 
 determining, based on the prediction provided for a particular component of the plurality of components generated for a first particular video content segment of the first sequence of video content segments, that the particular object associated with the subject matter is present in the first particular video content segment; 
 responsive to determining that the particular object associated with the subject matter is present in the first particular video content segment, generating a first identifier for the first particular video content segment that identifies the particular object associated with the subject matter is present in the first particular video content segment; 
 generating, based on the first identifier, a first redacted video content segment for the first particular video content segment, wherein the particular object associated with the subject matter that is present in the first particular video content segment is absent from the first redacted video content segment; and 
 reassembling the first sequence of video content segments with the first particular video content segment replaced with the first redacted video content segment to generate a first portion of redacted video content for the media content 
 for the second portion of video content: 
 segmenting the second portion of video content into a second sequence of video content segments; 
 processing each video content segment of the second sequence of video content segments using the multilabel classification machine-learning model to generate a second feature representation comprising the plurality of components for the video content segment, wherein each component of the plurality of components represents the particular object associated with the subject matter and provides a prediction as to whether the particular object associated with the subject matter is present in the video content segment; 
 determining, based on the prediction provided for a particular component of the plurality of components generated for a second particular video content segment of the second sequence of video content segments, that the particular object associated with the subject matter is present in the second particular video content segment; and 
 responsive to determining that the particular object associated with the subject matter is present in the second particular video content segment, 
 generating a second identifier for the second particular video content segment that identifies the particular object associated with the subject matter is present in the second particular video content segment; 
 generating second metadata for the media content, wherein the second metadata comprises the first identifier and the second identifier; and 
 responsive to the first request, providing the media content comprising the first portion of redacted video content, wherein playing the media content comprising the first portion of redacted video content results in the particular object associated with the subject matter that is present in the first particular video content segment not being displayed and the particular object associated with the subject matter that is present in the second particular video content segment being displayed. 
 
     
     
       8. The system of  claim 7 , wherein generating the first redacted video content segment comprises at least one of removing the particular object associated with the subject matter from the first particular video content segment, replacing the particular object associated with the subject matter in the first particular video content segment, or altering the particular object associated with the subject matter in the first particular video content segment. 
     
     
       9. The system of  claim 7 , wherein the first feature representation comprises a bounding box defined by a point, width, and height position for the particular object associated with the subject matter in the first particular video content segment and generating the first redacted video content segment comprises: performing, based on the particular object being located within the bounding box, at least one of removing the particular object associated with the subject matter from the first particular video content segment, replacing the particular object associated with the subject matter in the first particular video content segment, or altering the particular object associated with the subject matter in the first particular video content segment. 
     
     
       10. The system of  claim 7 , wherein generating the first redacted video content segment comprises replacing one or more frames of the first particular video content segment comprising the particular object associated with the subject matter with one or more replacement frames comprising altered content. 
     
     
       11. The system of  claim 7 , wherein the operations further comprise: receiving a second request for the media content, wherein the second request identifies the second individual; identifying, based on the second request identifying the second individual, the second identifier in the second metadata for the media content; segmenting the second portion of video content into the second sequence of video content segments; generating, based on the second identifier, a second redacted video content segment for the second particular video content segment, wherein the particular object associated with the subject matter that is present in the second particular video content segment is absent from the second redacted video content segment; reassembling the second sequence of video content segments with the second particular video content segment replaced with the second redacted video content segment to generate a second portion of redacted video content for the media content; and responsive to the second request, providing the media content comprising the second portion of redacted video content, wherein playing the media content comprising the second portion of redacted video content results in the particular object associated with the subject matter that is present in the first particular video content segment being shown and the particular object associated with the subject matter that is present in the second particular video content segment not being shown. 
     
     
       12. The system of  claim 7 , wherein determining that the particular object associated with the subject matter is present in the first particular video content segment is based on the prediction provided for the particular component satisfying a threshold. 
     
     
       13. A non-transitory computer-readable medium having program code that is stored thereon, the program code executable by one or more processing devices for performing operations comprising:
 obtaining first metadata for media content comprising first portion of audio content of a first individual involved in the media content and second portion of audio content of a second individual involved in the media content, wherein the first metadata identifies (1) the first portion of audio content is associated with the first individual and (2) the second portion of audio content is associated with the second individual; 
 receiving a first request for the media content, wherein the first request identifies the first individual; 
 for the first portion of audio content: 
 segmenting the first portion of audio content into a first plurality of audio content segments; 
 processing each audio content segment of the first plurality of audio content segments using a multilabel classification machine-learning model to generate a first feature representation comprising a plurality of components for the audio content segment, wherein each component of the plurality of components represents a particular keyword associated with subject matter and provides a prediction as to whether the particular keyword associated with the subject matter is present in the audio content segment; 
 determining, based on the prediction provided for a particular component of the plurality of components generated for a first particular audio content segment of the first plurality of audio content segments, that the particular keyword associated with the subject matter is present in the first particular audio content segment; 
 responsive to determining that the particular keyword associated with the subject matter is present in the first particular audio content segment, generating a first identifier for the first particular audio content segment that identifies the particular keyword associated with the subject matter is present in the first particular audio content segment; 
 generating, based on the first identifier, a first redacted audio content segment for the first particular audio content segment, wherein the particular keyword associated with the subject matter that is present in the first particular audio content segment is absent from the first redacted audio content segment; and 
 reassembling the first plurality of audio content segments with the first particular audio content segment replaced with the first redacted audio content segment to generate a first portion of redacted audio content for the media content; 
 for the second portion of audio content: 
 segmenting the second portion of audio content into a second plurality of audio content segments; 
 processing each audio content segment of the second plurality of audio content segments using the multilabel classification machine-learning model to generate a second feature representation comprising the plurality of components for the audio content segment, wherein each component of the plurality of components represents the particular keyword associated with the subject matter and provides a prediction as to whether the particular keyword associated with the subject matter is present in the audio content segment; 
 determining, based on the prediction provided for a particular component of the plurality of components generated for a second particular audio content segment of the second plurality of audio content segments, that the particular keyword associated with the subject matter is present in the second particular audio content segment; and 
 responsive to determining that the particular keyword associated with the subject matter is present in the second particular audio content segment, generating a second identifier for the second particular audio content segment that identifies the particular keyword associated with the subject matter is present in the second particular audio content segment; 
 generating second metadata for the media content, wherein the second metadata comprises the first identifier and the second identifier; 
 responsive to the first request, providing the media content comprising the first portion of redacted audio content, wherein playing the media content comprising the first portion of redacted audio content results in the particular keyword associated with the subject matter that is present in the first particular audio content segment not being heard spoken by the first individual and the particular keyword associated with the subject matter that is present in the second particular audio content segment being heard spoken by the second individual. 
 
     
     
       14. The non-transitory computer-readable medium of  claim 13 , wherein generating the first redacted audio content segment comprises at least one of removing the particular keyword associated with the subject matter from the first particular audio content segment, replacing the particular keyword associated with the subject matter in the first particular audio content segment, or altering the particular keyword associated with the subject matter in the first particular audio content segment. 
     
     
       15. The non-transitory computer-readable medium of  claim 13 , wherein generating the first redacted audio content segment comprises: processing the first particular audio content segment using a rules-based model to generate a redaction action, the redaction action involving at least one of removing the particular keyword associated with the subject matter from the first particular audio content segment, replacing the particular keyword associated with the subject matter in the first particular audio content segment, or altering the particular keyword associated with the subject matter in the first particular audio content segment; and performing the redaction action to generate the first redacted audio content segment. 
     
     
       16. The non-transitory computer-readable medium of  claim 13 , wherein the operations further comprise: receiving a second request for the media content, wherein the second request identifies the second individual; identifying, based on the second request identifying the second individual, the second identifier in the second metadata for the media content; segmenting the second portion of audio content into the second plurality of audio content segments; generating, based on the second identifier, a second redacted audio content segment for the second particular audio content segment, wherein the particular keyword associated with the subject matter that is present in the second particular audio content segment is absent from the second redacted audio content segment; reassembling the second plurality of audio content segments with the second particular audio content segment replaced with the second redacted audio content segment to generate a second portion of redacted audio content for the media content; and responsive to the second request, providing the media content comprising the second portion of redacted audio content, wherein playing the media content comprising the second portion of redacted audio content results in the particular keyword associated with the subject matter that is present in the first particular audio content segment being heard spoken by the first individual and the particular keyword associated with the subject matter that is present in the second particular audio content segment not being heard spoken by the second individual. 
     
     
       17. The non-transitory computer-readable medium of  claim 13 , wherein processing each audio content segment of the first plurality of audio content segments using the multilabel classification machine-learning model to generate the first feature representation comprises:
 processing the audio content segment using a speech-to-text engine to generate a text representation of the audio content segment, the text representation comprising a sequence of words spoken in the audio content segment; and 
 processing the text representation using the multilabel classification machine-learning model to generate the first feature representation.