Skip to main content
Version: V11

AI content processing in VIDIZMO

Overview​

VIDIZMO AI content processing automates the analysis and enrichment of media content. Using AI and machine learning services, it makes video, audio, image, and document content searchable, accessible, and compliant with organizational policies.

You can configure individual processing features according to your requirements. Each capability functions independently, although some rely on the output of other services.

Key capabilities​

VIDIZMO AI content processing involves:

  • Optimize media for playback: Convert video and audio into formats that are compatible across devices and browsers.
  • Extract text and metadata: Automatically generate transcripts, captions, and structured metadata from audio, video frames, and documents.
  • Detect and redact sensitive information: Identify personally identifiable information (PII) and other confidential content for compliance.
  • Generate summaries and chapters: Automatically create navigable summaries, chapters, and captions to enhance content consumption.
  • Translate content: Support multilingual audiences by translating documents, images, and media text.
  • Enable AI-powered search and retrieval: Transform content into embeddings to make it discoverable for AI-driven queries.

Configurable workflow​

VIDIZMO AI content processing is modular:

  • Each feature can be enabled or disabled independently.
  • Dependencies between capabilities are managed automatically.
  • Processing occurs in stages, ensuring efficiency and scalability.

How AI content processing works in VIDIZMO​

VIDIZMO AI content processing relies on a coordinated set of services that handle content analysis, enrichment, and preparation for search and playback. When a media file is uploaded, the VIDIZMO Indexer determines which processing features are enabled, orchestrates the workflow, calls the relevant AI services, and collects results for storage and downstream use.

Key components​

Component Purpose
VIDIZMO Indexer Orchestrates content processing, coordinates AI services, and extracts text, metadata, and timed data from uploaded media.
Speech-to-text service Converts spoken audio into text transcripts for videos and audio files.
Object detection service Identifies objects and faces in video frames and images to support indexing and redaction.
OCR service Extracts text from images and documents using optical character recognition.
LLM service Generates summaries, chapters, and translations using large language models.
Embedding service Converts extracted text into vector embeddings—numerical representations that capture semantic meaning—for AI-powered search and retrieval.
Vector database Stores embeddings and enables similarity search for efficient content retrieval.

Workflow execution​

VIDIZMO AI content processing is modular and parallelized where possible:

  • Features that do not depend on each other, such as transcription and object detection, can run simultaneously.
  • Features with dependencies, like summarization which requires transcription output, wait for the required data before executing.
  • The system ensures that outputs are stored, indexed, and available for downstream processes, including AI-driven search and content retrieval.

This approach maximizes efficiency, scalability, and content accessibility while maintaining the integrity of the processing workflow.

Supported content types​

Content Type Available Processing Features
Video files Transcoding, transcription, translation, object detection, face detection, OCR, summarization, chapter generation, PII redaction
Audio files Transcoding, transcription, translation, speaker diarization, PII detection, audio redaction
Image files OCR, object detection, face detection, translation, PII redaction
Document files OCR, translation, PII detection, document redaction

Processing features​

Transcoding​

When a video or audio file is uploaded to VIDIZMO, the system first checks whether it is in a compatible format, such as .mp4, and playable in HTML5. If compatible, thumbnails are generated automatically, and the content is immediately available for playback.

For files that require conversion, VIDIZMO performs encoding and decoding to ensure compatibility across devices and browsers. Videos are typically encoded into multiple bitrates 240p, 480p, 720p, and 1080p and output in streaming formats including Microsoft Smooth Streaming, HLS, MPEG4, and WebM.

Adaptive bitrate streaming allows the VIDIZMO player to select the appropriate quality based on device type and network conditions, ensuring smooth playback across all scenarios.

Audio analysis​

VIDIZMO processes the audio track of video and audio files to convert spoken content into searchable text. The system extracts audio, identifies spoken words, detects source languages including multilingual speech and generates time-coded transcripts synchronized with the media timeline.

Speaker diarization identifies distinct voices and labels dialog segments. Transcripts can also be translated into target languages, preserving the time-coded structure. Additionally, personally identifiable information (PII) can be detected in the transcript and redacted in the audio.

Object detection, face detection, and OCR​

VIDIZMO analyzes video frames and images to detect objects, faces, and text. Frames are processed by detection models to identify objects, faces, and other configured categories. Detected elements can be redacted if needed.

For documents and images, OCR extracts text while preserving positional information. Extracted text can then be used for indexing, search, or downstream AI processing. Sensitive visual information can also be identified and redacted automatically.

Automatic chapter generation and summarization​

Using large language models (LLMs), VIDIZMO generates summaries and chapters from completed transcripts. Summarization condenses content into key topics, while chapter generation identifies logical segments, assigns chapter titles, and integrates markers into the video timeline for easy navigation.

Keyword redaction using regex patterns​

Administrators can define custom patterns and keyword lists to detect sensitive information in content. During processing, text from transcripts or OCR output is scanned for matches, triggering redaction in audio, visual, or document content. This enables automated compliance with organizational policies.

Document and image translation​

VIDIZMO translates text in documents and images while preserving layout and formatting. OCR output is used to identify text blocks, which are sent to the LLM for translation. The translated content is then regenerated into the original document or image, maintaining positioning, styling, and readability for target languages.

Feature dependencies​

Some processing features require output from other features before they can run.

Feature Requires
Summarization Transcription
Chapter generation Transcription
Audio PII redaction Transcription
Visual PII redaction OCR
Document and image translation OCR

Independent features: Transcoding, transcription, translation, OCR, and object detection can run without dependencies.

Example scenario​

A law enforcement agency uploads body camera footage to their VIDIZMO portal. The portal is configured with transcription, face detection, OCR, and PII detection using custom regex patterns for badge numbers and case identifiers.

When the footage is uploaded, the system transcodes the video into multiple streaming formats and generates thumbnails. Audio analysis extracts the audio track, detects the spoken language, and generates a time-coded transcript with speaker labels. Face detection processes video frames and identifies all faces with bounding box coordinates. PII detection scans the transcript and OCR output for badge numbers and case IDs matching the configured patterns. The system then generates a redacted version with detected faces blurred, on-screen text containing PII redacted, and audio segments containing PII muted.

The result is a processed video with searchable transcripts, speaker identification, redacted faces, and protected sensitive information ready for case review and compliance requirements.

Did you find this article helpful?