Bill Gates and Nathan Myhrvold, the technology duo who brought us such classics as the Hurricane Suppression System for the Gulf of Mexico, are back with another idea: A device or app that can automatically create a customized video snippet from any random selection of text.

Nathan Myhrvold and Bill Gates check out new inventions for the toilet
Nathan Myhrvold and Bill Gates at a Gates Foundation event last year.

We know this, of course, because they filed for a patent on the approach.

The newly surfaced patent application, “Autogenerating Video From Text,” describes a device that can scan text on a page, read and analyze the text, understand what a given sentence is saying and automatically create some kind of video (or series of pictures) based on that text … with features that tailor the experience to a user’s preferences.

Gates and Myhrvold are two of 10 inventors listed on the patent filing — a telltale sign that this was a product of one of Intellectual Ventures’ brainstorming sessions. Intellectual Ventures leader Myhrvold, the former Microsoft technology chief, is involved in a variety of ventures with Gates, and has regularly included him in these sessions.

So what are the potential applications of “Autogenerating Video From Text”?

Here’s an example from the filing, “According to one contemplated scenario, a student is assigned a reading assignment. To make the assignment more interesting, the student may use his or her mobile phone to take a picture of a page of the textbook. The systems and methods described herein may then generate a synthesized image sequence of the action occurring in the text. Thus, rather than simply reading names and dates, the student may see soldiers running across a battlefield.”

Another actual example from the filing illustrates the personalization features: Someone reading Shakespeare could scan a section of text to create a video depicting one of the scenes — and the system could be set to “insert family members into the video clip instead of the typical characters.”

The GeekWire reader who tipped us off to this patent filing got very creative, and had some fun envisioning how Gates would use this …


The patent application was submitted in January 2012 and made public a couple weeks ago, on July 25.

The whole idea of generating videos or images from any random selection of text seems like a giant stretch, or at least a very difficult computer science problem. I’ve asked Intellectual Ventures if there’s a working prototype. Stay tuned.

Like what you're reading? Subscribe to GeekWire's free newsletters to catch every headline


  • galenward

    It is incredibly frustrating and stupid that these guys can sit around and come up with an idea, patent it with no actual working code or prototype, and then wait for it to become reality.

    This is what’s wrong with the patent system.

    • quesrty

      Strongly agree. Enough with the patenting of ideas. I can have an idea for a solar powered flying car, so can I get a patent?

    • Michelle

      This is a published application. It is not an actual patent. The patent system did what it was supposed to do, publish what was filed. No blame to be had on the USPTO for this one.

  • Mark Illing

    This idea has been the topic of research for communicating with individuals with aphasia. Those with aphasia cannot process language as others do, but visual communication such as pictures and video is effective. It will be interesting to read the patent to see what they are adding that does not already have prior art. I wish them well because doing this is not easy.

  • Mark Illing

    Do they lose points for misspelling “past”?

    • Punned_It

      Perhaps they just forgot a comma. They ran, passed a church and a school, to the candy store. A bit clumsy, a ‘they’ in the subordinate clause would be preferable for clarity, but it would be acceptable.

  • johncdvorak

    This is ludicrous. Next: turning farts into text then into video.

    • Dmitry Kirsanov

      There is a prior art for that. Twilight book and the movie.

  • Punned_It

    What would be more useful would be the reverse – taking video and generating text with descriptions and transcriptions of dialogue. There are many people with visual impairments; described video is expensive and takes time to produce.

  • JB

    This is not so difficult to accomplish honestly. Where they mention ‘autogeneration’ is probably more like ‘search and return results’. We do it today with simple text search that returns youtube videos. The OCR of the text is a faceted search parameter. Enough videos, media, etc that is tagged in the library and the system will pretty easily serve up the right one or combination.

  • Alfonzo

    That’s not so new at all. I’ve first heard a very same idea in 1989 (or so) from a futurist/cartoonist. He would imagine writing a script and have the basic cartoon sequence to be auto-generated. I guess there was a sort of prototype code as well.

  • Ian Mercer

    These ‘geniuses’ have absolutely no clue as to the meaning of art, and definitely no appreciation for its value nor that of the human brain to be the apex of imagination. They’re focused on creating “value” based solely on having things be ‘entertaining.’ Technology doesn’t ‘fix’ boredom, it can actually make you incapable of escaping it!

Job Listings on GeekWork