Next chapter in learning and deploying AI tools: a tool to summarise arbitrary-length PDFs. Very useful for dry, long reports, legal judgements, etc. You could use it on a book, but it will be rather … dry?

Code at https://github.com/langabi/RecursiveSummarizer. It’s a simple (and not particularly elegant) Python script, but also a MacOS finder shortcut (right click on any PDF to summarise it, like a super-power on my computer!) Needs minor technical skill, and a (free) OpenAI.com API key.

The main point of this is to explore how to work around one of the key constraints of current Large Language Models like GPT-3: the limited size of the prompt. GPT-3 current version can, at time of writing, handle around 4000 tokens, or around 3000 words in the combined input and output. So, if you have a document bigger than this, chunking and recursive summaries are one way to go. It produces a decent result, at the risk of the summary losing a little of the overarching theme of a document.

More generally, a key technique to avoid AI models from, well, making things up, is to enrich the prompt with information relevant for an accurate answer to the prompt question. The limited prompt size is a constraint here too — you might find yourself using AI techniques like embedding to find exactly and only the needed content.

But certainly, I expect that there will be a lot of innovation in the next few months in how to give AIs working memory (long and short term), through things like architectural changes (bigger prompt windows), embedding-driven long-term memory, and maybe using the bots themselves to create a summary of a current conversation, that is repeated at the start of each new prompt window. And no doubt much more!

We just reduced new product copywriting time from 40 minutes to 15 minutes, with our first internal AI-powered tool. Here’s how I’m thinking about AI as the “spreadsheet for copywriting”.

Technically, the implementation relies on:

• Enriching the “prompt” by replacing product- and order-specific references with all the information we have available, before passing the (much bigger) prompt to GPT-3, so that the AI is working on complete, current product and order information. Currently we’re just matching product/order IDs, but a more complete implementation will also used named entity extraction to find products, and then traditional search or embedding-based search to match and retrieve the matching products
• Adding some boilerplate to the prompt about tone of voice, and instructing it to respond “I don’t know” when it is not confident in being able to answer the prompt.

Without this, we found (as have many others) that the AI makes up very plausible but completely fictional details.

For now, we’ve released the tool in an internal “chat bot”-style interface, but are looking at a Google Sheets function, to allow generating text for each of a list of products in response to a standard prompt — very useful for newsletter creation. Regardless, human review and editing is critical for all our use cases.

So far, we’re seeing fastest adoption by our copywriters, especially for listing new products. Faithful to Nature’s team, for example, spends hours reviewing each and every ingredient with our suppliers, before listing a product. This isn’t changing, but the process to convert that information into a standard, compelling product description, has dropped from 40 minutes to 15 minutes per product.

So where to from here? (I’m asking, and so, to be honest, are our copywriters).

I keep returning to a podcast by Planet Money on the impact of the spreadsheet on finance and accounting. Tl;dr:

• spreadsheets used to be physical paper, calculated with hand calculators, laboriously
• digital spreadsheets (Lotus 1-2-3, Excel) completely changed this, and away went all that human work
• but, since then, finance employment has grown substantially
• because spreadsheets changed from a reporting tool, to a scenario planning tool: what would happen if we opened another store? Changed our margin? So finance itself become far more strategic.

This is how I’m thinking about these AI copywriting tools: by replacing the “hand-held calculator” of having to write every single word by hand, we will achieve two things:

• Unlock opportunity for copy, that was previously marginal. Ideally we’d have a page on our site for every possible combination of interests and questions for every small segment of customers, but that’s a lot of work. If the work halves, a lot more become possible — and indeed, we’re already working on newly-opened opportunities.
• Make copywriting a strategic “what if” function. Currently, the marketing flow is linear: come up with a campaign concept, choose a tone of voice, identify copy requirements, write the copy, send to customer. This can become iterative: if 90% usable copy is generated instantly, we can experiment with different concepts based on how they turn out, before finalising the brief.

Clearly, elevating copywriting from  execution to what-if strategy is very applicable elsewhere, for example, in the legal industry.

If you’ve deployed AI tools yet, what have your experiences been?