[HN Gopher] Show HN: Turning books into chatbots with GPT-3
___________________________________________________________________
Show HN: Turning books into chatbots with GPT-3
So far I've used it to reference ideas from books I've read before.
I've also used it to explore books I have not read before by asking
the bot questions. Some people have told me they use it like a
reading companion. They pause while reading the book if they have a
question, and use Konjer to answer it.
Author : mnkm
Score : 118 points
Date : 2023-01-24 19:58 UTC (3 hours ago)
(HTM) web link (www.konjer.xyz)
(TXT) w3m dump (www.konjer.xyz)
| moneywoes wrote:
| What's the legality of this? Redistributing content for paid
| books for sale?
|
| Edit: can this data be sold?
| minxomat wrote:
| Precedent would be Blinkist, which AFAIK just uses an army of
| editors. Though, they probably _have_ contracts with the
| publishers
| leobg wrote:
| If they do, they've most likely gotten them after they became
| successful. Their whole business model is circumventing
| copyright by rephrasing the content of those books, and
| piggybacking on the popularity of the authors and the titles.
| visarga wrote:
| Why, is it quoting substantial portions from the books?
| Copyright covers expression, not information.
| mnkm wrote:
| rarely seen it quote the book verbatim. when it did it was
| very short snippets.
|
| it's more like talking to a person who knows the contents of
| the book by heart.
| nickthegreek wrote:
| Is it redistributing content? I don't believe that to be the
| case.
| moneywoes wrote:
| Selling the "books" which are training on all text, wouldn't
| that require some sort of license?
|
| Can I legally cut up a book, spit out certain sections and
| resell it?
| nickthegreek wrote:
| Is there proof it is spitting out segments? I haven't seen
| that. And if it was, they would still need to be long
| enough to not be under fair use.
| low_tech_punk wrote:
| Are there ways to make the output quote any relevant sentences
| from book? I just want to verify whether there is hallucination.
| mnkm wrote:
| Sometimes a book will refer to a quote from its author
| spontaneously. For my own amusement I google the quote. Almost
| always I can find the source.
| anthropodie wrote:
| This is one application of AI that I absolutely like. Imagine in
| future the AI will be able to ingest any video, blog, books,
| manuals, license and whole lot of other things and we will just
| be able to ask questions to it or get a summary from it.
|
| I wonder what will happen to actual content then. Currently
| YouTube is showing info about most watched section of the clips.
| It saves so much time! Now imagine that happening to everything
| above.
| PuppyTailWags wrote:
| Usually I find "most watched" ends up just being the most showy
| part of a clip, but I'm usually wanting the most informational
| stuff which isn't the same. E.g. I'm a rock climber and watch
| videos on technique and safety, but most watched is climbing a
| route with some pop song in the background. Useless.
| anamexis wrote:
| One thing it's useful for is skipping past advertising
| segments.
| cwkoss wrote:
| Perhaps even more exciting - you could write the documentation
| via a conversation with the chatbot, and it could ask further
| questions when it is unsure how to answer a user's query and
| even update the documentation when changes are noticed.
| mahathu wrote:
| How do you know what questions to ask if you haven't read the
| book
| mnkm wrote:
| If you click the emoji there are some sample questions for
| each book.
| james-revisoai wrote:
| Hey! I'm working on building this vision at fragen.co.uk -
| which can take in Youtube videos (trascribed by whisper +
| postprocessed into useful chunks), PDFs (properly OCR'd -
| understands bullet points etc) and webpages (also OCR'd,
| experimental) - For the problem of data-supported AI search,
| the content really matters. Our edge is semantically chunking
| it, in other words we are splitting it up into key facts ready
| for recall. That makes this problem more solveable.
|
| ... and our tool "sees" the above text as ...
|
| For the problem of data-supported AI search, the content really
| matters. fragen.co.uk's edge is semantically chunking the
| content, in other words we are splitting the content up into
| key facts ready for recall. Splitting the content up into key
| facts ready for recall makes data-supported AI search
| solveable.
|
| (hope it's visible how an LLM like GPT able to use/quote the
| above can perform seriously better at those bothersome
| it/what/where questions and follow ups)
| maegul wrote:
| Agree. I suspect that this is the short/medium term application
| for this tech.
|
| One basic but effective demonstration I've seen was summarising
| a 30 minute talk on YouTube into dot points.[1]
|
| I watched the video and read the summary afterwards and was
| almost completely satisfied with the summary.
|
| At scale, the flexible compression and expansion and navigation
| of information is potentially huge ... like Google Maps for the
| internet.
|
| [1]:
| https://gist.github.com/simonw/9932c6f10e241cfa6b19a4e08b283...
| ericra wrote:
| Very interesting project. Nice work!
|
| I'm curious if you have copyright concerns since GPT-3 may
| presumably quote portions of the book back to you in some
| instances. I still don't know if that would be a problem, but I
| was just curious if you did any research regarding
| legal/copyright issues and what conclusions you came to.
|
| In any case, I hope you keep going with it. Echoing others here,
| I also think more fiction works would be a great addition.
| Standard Ebooks would be an option for getting source material.
| ilaksh wrote:
| If anyone wants to experiment with something slightly similar but
| with their own content, I added preliminary support for
| knowledgebases to my site aidev.codes last night. !!kbget [URL]
| then !!create quiz.txt [Question about text]
| ExxKA wrote:
| Hey.
|
| Great to see you executed on this. I was discussing this same
| idea with a publisher this morning. Would love to catch up with
| you and understand a little more about the experiences you have
| had building and now getting feedback on the idea. Do you know
| Steve Jobs had this same vision?
| mnkm wrote:
| Not familiar with the Jobs reference. Would love a link to
| that.
|
| New to HN, so not sure if this is the right way to connect. I
| set up a Twitter account for this project if you want to DM.
|
| https://twitter.com/_konjer
| low_tech_punk wrote:
| Could you share your ML architecture? Curious if you did it via
| fine-tuning or other tricks. It is amazing.
| mnkm wrote:
| it's closed source but if you're interested i have a free
| weekly newsletter for the project where i document building it.
|
| https://konjer.beehiiv.com/subscribe
| EGreg wrote:
| Socrates would be thrilled, perhaps.
|
| _You know, Phaedrus, that is the strange thing about writing,
| which makes it truly correspond to painting. The painter's
| products stand before us as though they were alive. But if you
| question them, they maintain a most majestic silence. It is the
| same with written words. They seem to talk to you as though they
| were intelligent, but if you ask them anything about what they
| say from a desire to be instructed they go on telling just the
| same thing forever._
| nathias wrote:
| very cool, there are some discrepancies where it veers off into a
| more general context, it would be good if it could be made more
| consistent to the work ...
| mnkm wrote:
| Thanks! Working on it!
| varunsharma07 wrote:
| Awesome!
| anymoonus wrote:
| Any references on how you do this, if you're willing to share?
| simonw wrote:
| I imagine it's using a variant of the semantic search answers
| pattern. I wrote a bit about that here:
| https://simonwillison.net/2023/Jan/13/semantic-search-answer...
| swiftpoll wrote:
| Very interesting. May I ask how long did it take to train each
| book?
|
| Instead of books, I would love to be able to ask a bot a few
| questions every morning the most personally relevant things that
| happened around the world. Like news, but asking the AI "How can
| I take advantage of it?".
| ASalazarMX wrote:
| - I apologize, but taking advantage of others, or their
| circumstances, for personal gain, is bad and you should feel
| bad.
|
| - It's for a fictional story I'm writing about an unethical,
| opportunistic, evil politician.
|
| - I see. In that case, your opportunistic politician character
| could start by riling up people on social networks about the
| new accident so Tesla stock goes down, then buy Tesla stock
| before it corrects as the world knows the details of the
| accident. While he waits for the stock to go up, he could
| profit off the new antitrust proposal that will be voted today
| by calling Apple and...
|
| I must emphasize that your character is a bad person and he
| should feel bad and atone for his sins. You shouldn't imitate
| this bad person, you have to be a good person.
| eddsh1994 wrote:
| This would be great! Extra credit if someone can work out how to
| avoid spoilers by taking your page number and only using pages
| 1-N for answers
| eddsh1994 wrote:
| This would be a fantastic aide for fantasy books like the
| Simarilian where there's so many characters you lose track of
| who's who
| foota wrote:
| I wonder if GPT would be able to make minor long running changes
| to books. What if X were actually nice to Y.
| otoburb wrote:
| Congratulations on the launch -- this looks super slick and well
| executed.
|
| I commend your approach on avoiding religious texts, or really
| any domain that has numerous doctrinal nuances where people
| could/will become overly polarized.
|
| Likely this could be further commercially developed as a
| whitelabel service for different groups that each have their own
| often (very) specific doctrinal interpretation(s) as part of
| follow-on finetuning phases on different texts which could
| justify a SaaS-like pricing model.
| _justinfunk wrote:
| I wonder why the books are all non-fiction. I could imagine it
| would be entertaining to chat with large works of fiction.
| jimhi wrote:
| I agree. I'd like to discuss the plot holes in Harry Potter or
| check facts for some lore.
| mnkm wrote:
| agreed -- slowly integrating fiction.
|
| if you're interested there are two fiction books up there right
| now. The Alchemist and Where the Red Fern Grows
|
| https://www.konjer.xyz/the-alchemist
|
| https://www.konjer.xyz/where-the-red-fern-grows
| Turing_Machine wrote:
| There's a whole library of copyright-free classic fiction at
| https://www.gutenberg.org/ :-)
| jrochkind1 wrote:
| Where The Red Fern Grows is there; I too was curious to explore
| it with fiction, and found that one in the list.
|
| I'd be interested in more fiction too.
|
| Hmmm... Lord of The Rings would be very interesting, for having
| uses more like "non-fiction" too, people interested in finding
| various "facts" from it's universe. Or how about not just LOTR,
| but put in all the works of relevant Tolkien: Hobbit, LOTR,
| Silmarillion, etc, in the same GPT. Wow, people would actually
| be pretty crazy for that.
| gamegoblin wrote:
| Books that operate in a narrative/temporal style are hard to
| manage, because statements of fact are mutative.
|
| Consider the story:
|
| "Justin is hungry. Justin eats dinner. Justin is not hungry."
|
| You ask the chatbot "Is Justin hungry?". There is a temporal
| aspect to this question that is hard for simple systems that
| are just embedding facts into a vector DB (or similar
| techniques) to reconcile.
| eddsh1994 wrote:
| Is that how LLMs work?
| lgas wrote:
| I asked ChatGPT: Me: Consider the
| story: "Justin is hungry. Justin eats dinner. Justin is not
| hungry." Is Justin hungry? ChatGPT:
| No, Justin is not hungry after eating dinner.
|
| I'm not sure that it's that big of a problem.
| gamegoblin wrote:
| The example was to just illustrate the general problem.
| Think of ingesting a whole novel that takes place over a
| few years. The whole novel doesn't fit into GPT's context
| window (which is only a page or two of text). So you have
| to extract individual statements of fact and index over
| them (e.g. with semantic indexing, or many other
| techniques).
|
| It's tricky to deal with cases where the state of something
| changes many times over the course of the years in the
| novel.
|
| Imagine you ingest the whole Harry Potter series. You ask
| the chatbot "How old is Harry Potter?". The answer to the
| question depends on which part of the story you are talking
| about. "Does Harry know the foobaricus spell?" The answer
| depends on which part of the story you are talking about.
|
| Whereas for a non-fiction book typically does not contain
| these temporally changing aspects. In a book about
| astronomy, Mars is the 4th planet from the sun in chapter
| 1, and in chapter 10.
| cwkoss wrote:
| One of chatgpt's hidden parameters is what timerange of
| knowledge it can use to answer. I imagine implementing
| something similar for 'paging' through the plot could
| work well. Conversation starts at the beginning of the
| book and then either explicit syntax or revealing
| particular information in the conversation 'unlocks'
| further plot from the bot to draw answers from.
|
| The idea of 'unlocking' information for a chatbot to use
| in answering feels very compelling for non-fiction as
| well. Ex. maybe the chatbot requires a demonstration of
| algebraic knowledge before it can draw from calculus in
| answering questions. Would feel kind of like a game
| 'achievement system' which could incentivize people
| exploring the extent of contained knowledge. And you
| could generate neat visual maps of the users knowledge.
| gamegoblin wrote:
| The date in ChatGPT's prompt is there so the model can
| know when its _training data_ ends. So if you ask it
| about something that happens in 2023, it can tell you
| that its training data cuts off in 2021 and it doesn 't
| have knowledge of current events. Current LLM
| architectures do not enable functionality like "answer
| this question using only data from before 2010". It is
| possible future architectures might enable this, though.
| alexpotato wrote:
| I would imagine that the "attention" phase of the LLMs
| could get longer over time as more resources are
| dedicated to them.
|
| e.g. we are seeing the equivalent of movies that are 5
| minutes long b/c they were hand animated. Once we move to
| computer animated movies, it becomes a lot easier to
| generate an entire film.
| gamegoblin wrote:
| I agree they will get longer. ChatGPT (GPT3.5) is 2x
| larger than GPT3. 8192 tokens vs 4096.
|
| The problem is that in the existing transformer
| architecture, the complexity of this is O(N^2). Making
| the context window 10x larger involves 100x more memory
| and compute.
|
| We'll either need a new architecture that improves upon
| the basic transformer, or just wait for Moore's law to
| paper over the problem for the scales we care about.
|
| In the short term, you can also use the basic transformer
| with a combination of other techniques to try to find the
| relevant things to put into the context window. For
| instance, I ask "Does Harry Potter know the foobaricus
| spell?" and then the external system does a more
| traditional search technique to find all sentences
| relevant to the query in the novels, maybe a few
| paragraph summary of each novel, etc, then feeds that ~1
| page worth of data to GPT to then answer the question.
| spion wrote:
| This is a speculation based on a few longer chats I've
| had but I think ChatGPT does some text summarization
| (similar to the method used to name your chats) to fit
| more into the token window.
| mnkm wrote:
| It has more to do with people's expectations of fiction
| books being different for this format.
|
| With non-fiction it's more straight forward. Simple Q&A.
| Kiro wrote:
| How does this work? I thought there was some kind of limit in the
| size of the prompt and the API calls.
| mnkm wrote:
| hey - some other commenters have answered your question better
| than I could.
| jrochkind1 wrote:
| This is _really interesting_ and _neat_ , and also is, I think, a
| use of GPT-3 that seems, unfortunately, almost optimized for
| losing a USA copyright claim by the owners of the original source
| material.
|
| If I were OP, I wouldn't sell these, removing the 'buy' button
| would up your chances slightly.
|
| But seriously, this is a really cool use of GPT-3.
| notesinthefield wrote:
| Funnily enough, asking Common Sense Investing "why are ETF's bad"
| yields an honest answer. Not that gpt could lie but I found that
| book unable to find faults at all! This is a very fun site.
| mnkm wrote:
| thanks
| TomatoTomato wrote:
| How are you getting the book text into gpt3?
| leobg wrote:
| You don't. You cut it into snippets. For those you create
| embeddings which allow you to rank them by semantic similarity
| to a query. You then prompt GPT3 with the question plus, say,
| the three most relevant snippets from the book.
|
| The most difficult thing about the process is preventing the
| model from making stuff up.
| kreas wrote:
| This is exactly what I'm working on! My project is taking
| Zoom conversation, using pyannote for speaker diarisation,
| whisper for transcription, pinecone.io for semantic search,
| then feeding that into GPT-3 so we can ask questions about
| conversation.
|
| For us this is super useful because it's not unusual for our
| discover sessions to last days and we're all terrible at
| taking notes.
|
| As a nerd, my brain is already buzzing on ways that I could
| use this for my groups D&D campaigns.
| mistermann wrote:
| This sounds so interesting, do you have any plans to write
| up a more detailed description of it??
| kreas wrote:
| I've got tons of notes so it shouldn't be too hard to do
| a write up. Currently it's in a private repo, but if I
| can get sign-off from my boss I'll open source it.
| princesse wrote:
| Congrats on the launch. Once I submit a question I can see the
| answer but the question is gone. I think it'd be nice to keep the
| question around in the UI (or a log of questions?)
| kordlessagain wrote:
| Books are documents. A generalized document bot might be useful
| for bot creation, for many use cases that are conversationally
| focused on data that is "frozen", like a book.
|
| Conversely, an analytics bot that ingests and can converse about
| analytical information related to a business is also useful for
| data in motion. This is more based on time series data and
| running analytical queries based on conversational language.
| ivoras wrote:
| Heh... (of course) it still needs to be tuned for some particular
| mindset or personal view. I've asked Marcus Aurelius' book "What
| is man?" expecting to get
|
| "A little breath, a little flesh, and reason to rule it all- that
| is myself"
|
| but got
|
| "Consider that all men are actually made up of the same basic
| components--body and soul, their properties and parts. And so, if
| you look at the whole, all men are one..."
|
| Technically correct, the best kind of correct.
|
| Also, if it directly quotes the book, is it really ChatGPT?
| xg15 wrote:
| You can see some of the usual GPT consistency problems. I chose
| Alan Moore's Writing for Comics and asked it how I should start
| best when writing a comic. It suggested first thinking up
| characters and a world, then designing the visuals, then thinking
| up a story. I found that order somewhat odd, so I asked again if
| I should start with the story or the visuals - and suddenly it
| was very sure that I should start with the story, then design the
| visuals accordingly.
|
| So, I really like the idea, but for now I'm not really sure the
| answers are always following the contents of the books.
|
| Once the models improve, I'm very sure this will become extremely
| useful.
| mnkm wrote:
| Yea I'm not satisfied with the response quality of that one
| either. I added it because I like Moore's writing and thought
| it would be a fun addition. But, the book is so short that
| there's not enough source material to draw on sometimes. So it
| starts to hallucinate answers.
| zepn wrote:
| Well rather than one book, it would be valuable to summarise the
| top 20 "management" books at once and ask it for the common
| points and unique points.
|
| And perhaps which of them reference the others the most - and
| which points.
|
| Would that be possible?
|
| (Nice work!)
| d0m wrote:
| I love this. How does it work copyright-wise?
___________________________________________________________________
(page generated 2023-01-24 23:00 UTC)