Next chapter in learning and deploying AI tools: a tool to summarise arbitrary-length PDFs. Very useful for dry, long reports, legal judgements, etc. You could use it on a book, but it will be rather … dry?
Code at https://github.com/langabi/RecursiveSummarizer. It’s a simple (and not particularly elegant) Python script, but also a MacOS finder shortcut (right click on any PDF to summarise it, like a super-power on my computer!) Needs minor technical skill, and a (free) OpenAI.com API key.
The main point of this is to explore how to work around one of the key constraints of current Large Language Models like GPT-3: the limited size of the prompt. GPT-3 current version can, at time of writing, handle around 4000 tokens, or around 3000 words in the combined input and output. So, if you have a document bigger than this, chunking and recursive summaries are one way to go. It produces a decent result, at the risk of the summary losing a little of the overarching theme of a document.
More generally, a key technique to avoid AI models from, well, making things up, is to enrich the prompt with information relevant for an accurate answer to the prompt question. The limited prompt size is a constraint here too — you might find yourself using AI techniques like embedding to find exactly and only the needed content.
But certainly, I expect that there will be a lot of innovation in the next few months in how to give AIs working memory (long and short term), through things like architectural changes (bigger prompt windows), embedding-driven long-term memory, and maybe using the bots themselves to create a summary of a current conversation, that is repeated at the start of each new prompt window. And no doubt much more!
Paul Cook 