upatov

[upatov]

Building the World's First True Reverse Video Search Engine

While it may not be widely known, no true video search engine currently exists. Services that claim to offer "reverse video search" are, in fact, merely searching via static images or individual frames, and they are transparent about these limitations.

The reason for this is simple: searching entire videos is a highly complex and resource-intensive task. The technical challenges and costs associated with processing full-length videos have made it prohibitive—until now. 

At Comexp Research Lab, we have developed a new approach, based on the Theory of Active Perception (TAPe), that allows us to tackle these challenges with a fraction of the usual computational resources.

Leveraging TAPe, we’ve created an innovative solution capable of handling complex information processing problems in a more efficient manner. This breakthrough has enabled us to develop a new video search engine—essentially Google for videos—which we’ve named TAPe Reverse Video Search (RVS). In this article, we detail the journey toward building this pioneering technology and outline the progress we’ve made to date.

A Tangible Milestone: Creating the First True Reverse Video Search Engine

At Comexp Research Lab, our work has focused on developing services based on our proprietary video-by-video search technology, which mimics the efficiency of human perception. The TAPe model represents a significant departure from traditional search methods by utilizing a perceptual approach rooted in group theory.

In our discussions with peers, investors, and the general public, we typically delve into the Theory of Active Perception (TAPe) and present demonstrations that are conceptually straightforward. Yet, the feedback is often the same: “This is fascinating, but can you show us something more concrete?”

This year, we reached that milestone. We’ve launched a prototype of our video-by-video search engine. Although still in its early stages, the engine indexes videos much like how Google began by indexing text-based websites. In Google’s case, as the volume of indexed sites grew, so did its ability to deliver rapid, relevant search results. The same principle applies to video search, albeit with far greater challenges.

Indexing video content requires substantially more computational resources than indexing text. Even with modern technologies, the process remains slow, costly, and inefficient. For this reason, no major company—Google included—currently offers a fully realized video search engine that searches entire videos. This is where TAPe provides a significant advantage.

Revolutionizing Video Search with TAPe

Our search engine, powered by TAPe, enables users to search a vast archive of indexed video content to locate specific videos. The process begins by comparing the user’s video query against the indexed database and delivering the most relevant matches.

The Theory of Active Perception (TAPe) is a set of novel methods we’ve developed that fundamentally changes how information is processed. This approach allows us to achieve results that are orders of magnitude more efficient than conventional methods—using thousands of times less computational power, less time, and fewer resources overall.

Given the rapidly increasing volume of video content, we began by indexing feature films, documentaries, and TV series. As of now, our system has indexed 80,000 movies. This forms the foundation of our search engine, similar to how textual search engines require comprehensive indexing to be effective.

Additionally, we’ve expanded our capabilities to include television search. Our system tracks broadcasts from major global TV channels, allowing users to discover when and where specific video content, such as TV episodes, has aired. Our next major goal is to index YouTube content, which will significantly enhance the power of TAPe RVS.

Introducing ComexpBot: A Practical Application of TAPe Video Search

To facilitate the use of TAPe RVS and explore potential applications, we’ve developed a Telegram-based bot called ComexpBot. This tool allows users to search for films, TV series, and broadcasts by submitting video fragments instead of traditional text or image queries.

For example, a user might upload a brief clip or GIF, and the bot will quickly identify the corresponding film or series if it exists in our database. The bot returns detailed information, such as the title of the content, links to related websites (like IMDB), and even available trailers.

One of the most striking features of the bot is its ability to recognize videos from small, low-resolution snippets—sometimes as small as 260 pixels. This showcases the efficiency of TAPe’s video sequence processing, which significantly reduces the computational overhead compared to traditional frame-by-frame analysis.

The Underlying Technology: How TAPe Works

Unlike traditional computer vision techniques that rely heavily on convolutional neural networks (CNNs) and deep learning, TAPe employs a unique methodology. Rather than focusing on individual frames, TAPe processes sequences of frames—typically around 5 seconds of video—at once. This approach is counterintuitive but far more efficient than analyzing frames individually, especially considering that a 5-second video segment can consist of 120 to 300 frames.

Importantly, TAPe does not require pre-trained models, as most computer vision systems do. Instead, it learns in real time during the recognition process, much like human perception. This real-time learning enables TAPe to bypass many of the computationally expensive steps involved in traditional video processing. As a result, TAPe can extract the minimal number of features necessary to identify video content, leading to a significantly more efficient search process.

By creating a lightweight “cast” of each video—known as a tape-index—TAPe captures the essential characteristics of the content, which allows for fast and accurate searches. This method drastically reduces storage requirements and computational complexity.

Looking Forward: The Future of TAPe and Video Search

TAPe’s potential extends far beyond its current application in video search. While we are focusing on video recognition and analytics, the underlying technology has broader implications for fields such as artificial intelligence, machine learning algorithms, CPU and GPU development, autopilot systems, and real-time video analytics.

We are also planning to offer the TAPe Video Search API, which will enable researchers and enterprises to analyze vast amounts of video content more efficiently. Additionally, we are developing an extension of the TAPe API for website developers, making it accessible to a wider audience.

One of our most ambitious goals is to index YouTube content, beginning with the platform’s most popular videos. Although this represents only a small fraction of the total content on YouTube, it still amounts to a staggering 2,500 years' worth of video footage. We are confident that TAPe’s efficiency will allow us to tackle this challenge within a reasonable timeframe.

As the use of video content continues to grow exponentially, the demand for efficient, large-scale video search solutions will only increase. TAPe’s revolutionary approach positions it to play a key role in meeting this demand, providing a sustainable and scalable solution for video search in the digital age.