[HN Gopher] Reverse engineering yet another eBook format
___________________________________________________________________
Reverse engineering yet another eBook format
Author : Metalnem
Score : 316 points
Date : 2022-12-25 10:32 UTC (12 hours ago)
(HTM) web link (mijailovic.net)
(TXT) w3m dump (mijailovic.net)
| yellow_lead wrote:
| Can anyone tell me the status of Kindle DRM? I want to buy some
| kindle only books but don't own a kindle. Is this even doable?
| blackoil wrote:
| Not the answer you seek, but Kindle has apps for major
| platforms and a web based viewer.
| vrotaru wrote:
| The web based reader will refuse to display some books,
| unfortunately.
|
| In my case this was the sort of book than can be only
| realistically read on a big screen tablet, not on a Kindle.
|
| Well, I've shrugged and downloaded the pdf from libgen
| themadturk wrote:
| The problem is that the latest versions of the Amazon reader
| for PC and Mac don't download the books in a format
| compatible with Calibre's DeDRM scripts. This is compounded
| by Amazon flipping off the choice to always update the
| software automatically (yes, it's stated as "always update
| automatically" if checked), so the first thing you have to do
| when starting up is go into settings and make sure the
| checkbox is cleared.
| yellow_lead wrote:
| Still useful - thanks!
| markx2 wrote:
| There are plugins - a search will find them - you can use with
| Calibre to remove DRM.
|
| https://calibre-ebook.com/
| rdl wrote:
| I don't think there is an easy way to DeDRM the latest kindle
| format used by the Mac/pc and iOS reader, and there is no
| easy way to download all kindle books in your account for
| "transfer to device via usb" for e-ink devices all at once.
| I'm trying to archive my 4k or so kindle books (already did a
| similar number of audible using openaudible) because I don't
| trust Amazon to preserve access to them indefinitely.
| Leherenn wrote:
| You can however still download and use an older version of
| the kindle program (at least on windows), which download
| the files with the old DRM scheme.
|
| It's one by one though.
| themadturk wrote:
| You can find older Kindle versions for Mac as well
| without too much effort.
| kcartlidge wrote:
| Anecdotally, I've become aware that the Windows Kindle
| installer version 1.17 works fine for downloading your
| paid-for Kindle books and format-shifting them or making
| a personal back-up in a more open form. That information
| was current about a year or so ago; hopefully that
| version has not been hobbled since. Also, disable auto-
| updating.
|
| I strongly suggest you only use it for your own
| purchases. Authors need to heat and eat too.
| barnacled wrote:
| I am writing a book I intend to release commercially, and a good
| sized one at that (likely 1,500 pages+). I am absolutely
| stubbornly determined to release it as both print and DRM-free
| PDF.
|
| It'll get pirated anyway (if anybody cares enough to do so), so
| why bother punishing paying customers?
| tapia wrote:
| This is really an issue that should not exist. I have also bought
| textbooks only to be immediately disappointed by the fact that
| you have to use a specific app (like the adobe DRM thing... or
| just try to get your money back). Then you have to start figuring
| out how to remove the DRM and lose time with that. Sometimes the
| quality of the book is also pretty bad, e.g. they don't even
| include a digital table of contents, which should be essential in
| a digital copy of any book (this has happened to me with
| textbooks from Springer). In my case, this has lead to actually
| buying less textbooks and just pirate the books instead. I don't
| have the time to deal with all this customer-unfriendly behavior.
| hackernewds wrote:
| These are pretty poor excuses to cope with piracy
| tapia wrote:
| I think that the poor excuses are in the other side. Adding
| DRM to files to actively preventing the user to choose in
| which devices he/she wants to read the file and for how long
| the content will be available goes against the whole concept
| of "buying" a book. When I buy a book I expect to be able to
| open it in ten years in whatever device I'm using, running
| any OS that I want. That's why we have standard formats. And
| I also expect that the quality of a book for which I payed
| 50-90EUR for (because that is normally the price range) to
| have a good quality and at least a table of contents. A few
| occasions I had to manually add table of contents for books
| that otherwise would have been unusable to me.
| T3OU-736 wrote:
| "buying a book". You're not, though. Nor a film/movie. It
| is, at least in the US, treated as something along the
| lines of a temporary contract to view. And so... DRM is
| used to as a tool of enforcement of the contract term's
| expiration.
| inkblotuniverse wrote:
| Hence piracy is the better option.
| brutusborn wrote:
| If legal options aren't useable, the black market will fill
| the void. Once this is fixed, the dynamic swings back (like
| how torrenting became less popular since streaming services)
| You might find it immoral, but 99% of the world disagrees
| with you (e.g. ubiquity of illegal drug use worldwide)
| goosedragons wrote:
| One of my textbooks came with a digital copy. I would sometimes
| just use that on my iPad instead of lugging in the physical
| book to campus. I remember racking my brain all day on a
| problem that asked you to prove some result, wondering how the
| hell that was true only to get home and realize the digital
| version flipped the inequality...
| psychphysic wrote:
| Could have been a Dantzig moment![0]
|
| [0] https://www.snopes.com/fact-check/the-unsolvable-math-
| proble...
| charlieyu1 wrote:
| As a maths teacher who also have written a textbook, I have
| seen a drastic increase in errors in books compared to 5-10
| years ago. I don't know why, maybe the scheduler is tighter
| or proofreading is now too expensive.
| this-pony wrote:
| From what I heard, it's because Springer et al. hired
| editors in India to cut costs...
| ta988 wrote:
| The last springer book I've had is so poorly written that
| there is absolutely no way it was proofread. It is not just
| one mistake here or there. It looks like a quick draft with
| incomplete sentences, repetitions all over. Looks like
| those "cheap" packt books.
| smnrchrds wrote:
| Looks like someone sent
| "fundamentals_of_math-e4-v2-final-finalv3-final.docx" to
| the printer instead of "fundamentals_of_math-e4-v2-final-
| finalv3-final-final.docx"
| riedel wrote:
| I published a book with degruyter and there is no
| proofreading at all. No support to get e.g. the colours
| right for printing. They only give you some example
| books, a rough latex template and minor advice on content
| and audience. Once it is published they will ask you if
| you want to prepare a revised version. It was OK because
| e.g. my students get a free ebook version anyways via the
| university subscription and I guess it is ok for the rest
| to pay the overpriced but well printed book.
| azalemeth wrote:
| I've authored book chapters with Springer. I've used their
| LaTeX class and produced some nice looking chapters (imho).
|
| They get "typeset" by SPI in Chennai by a non maths
| speaker, non-native English speaker and come back with
| errors in both the equations and words and author queries
| like "equation missing but we follow tex".
|
| One professional mathematican I know in a suitably obscure
| area of number theory has started her own journal to get
| around it all. I'm in both the physics and medicine camps
| and have to put up with this bullshit. It's awful.
| jtbayly wrote:
| I have a $70 ebook (Amazon Kindle) of an academic work (not a
| textbook) and it is practically unusable. So many problems
| compared to the physical book. Never again.
| treeman79 wrote:
| With my adhd I learn programming drastically better on
| physical books. Stupid I know, but it helps drastically to
| see it on paper.
|
| Sadly it's either crazy expensive, or not even an option to
| get paper for newer frameworks
| ykonstant wrote:
| > Stupid I know, but it helps drastically to see it on
| paper.
|
| Not stupid at all. Myself, and every single colleague of
| mine that I know (pure mathematicians) have to print every
| non-trivial document that we _really_ want to study deeply.
| TillE wrote:
| I've found that a ton of older books published as ebooks were
| clearly just OCR'd and then not thoroughly edited. Definitely
| frustrating.
| Finnucane wrote:
| Handling complex material has never been a priority for the
| Kindle. To be fair, more recent iterations have improved
| support for things like tables and MathMl, but it's still a
| work in progress.
| cl3misch wrote:
| > My old instincts kicked in and I decided it would be more fun
| to hack and blog for a week than to waste any time dealing with
| the customer support.
|
| Don't get me wrong, I like reverse engineering as much as the
| next guy. But if you really are annoyed by such DRM tactics, you
| _should_ deal with their customer support to communicate your
| frustration. Otherwise, nothing will ever change.
| survirtual wrote:
| Since when has complaining to customer support as a retail
| consumer ever changed anything? Retail can cause change by
| using third party media channels and, by some miracle,
| generating enough attention that it makes business sense to
| respond.
|
| Enterprise customers are a different story.
| charcircuit wrote:
| This isn't a new ebook format. This is just a web viewer for epub
| files.
| viraptor wrote:
| > Downloading the files was super easy, barely an inconvenience
|
| I love how the zeitgeist / popular culture allows this expression
| to escape the original context and become popular, but not a
| silly meme.
| Chinjut wrote:
| What expression? What was the original context?
| edflsafoiewq wrote:
| https://www.urbandictionary.com/define.php?term=Super%20Easy.
| ..
| eesmith wrote:
| I was curious if there was an older history. A Google Book
| search for: "Super Easy" "Barely an Inconvenience" indeed
| found nothing before about 2018, and then a few books use
| that combined phrase.
|
| Most intriguingly, one is in Shatner's 2022 book "Boldly
| Go: Reflections on a Life of Awe and Wonder"
|
| > Two of my three daughters live within a couple of miles
| of my house, while the other lives about fifty miles away -
| super easy to visit, barely an inconvenience.
|
| Either he's really plugged into YouTubers or I can't help
| but think it comes from a ghostwriter.
| PaulHoule wrote:
| Those web viewers for e-Books make PDF look great. There is
| nothing I dread more at the library for my uni than seeing a book
| is "in the collection" but is only visible through a web viewer.
| It's like trying to read the book through a shaving mirror.
|
| (1) They try to slice the book into the thinnest possible salami
| so you are always click click clicking to navigate
|
| (2) You can't download the book to read on the bus
|
| (3) There is a general "evil" trend in the web to hide the
| content as much as possible. Cookie popups and "subscribe to our
| email newsletter" popups count, but hard-to-use navigation,
| <iframe> and friends all make their contribution. You'd think the
| people who make this stuff are paid more the less content is
| visible on the page.
|
| (4) and of course it seems like the people are paid the more you
| have to struggle to log in.
| rightbyte wrote:
| Ye. I find it insane that it is faster to look in the paper
| book register and find the correct page then search their
| crappy ebooks. I have timed it. The ebook experience is
| abyssal.
| RomanPushkin wrote:
| > "Access Duration: 84 Months"...
|
| > Despite all the warning signs, I went ahead and bought the
| ebook...
|
| > ...allow me to retire from having to write the tools that
| bypass your restrictions.
|
| You either agree to the rules, or you're breaking them. If you're
| breaking the rules, authors won't get compensated and we won't
| see other Human Kenetics books. This should be illegal.
|
| Personally, I would love to pay hundreds for a limited access to
| books I would like to read. But folks aren't inspired too much
| about writing books today that can make them a living, since
| there are pirates everywhere. They want free content, because
| "information should be free". No, it shouldn't. Pay for it.
| tucosan wrote:
| This is a ridiculous take.
|
| There were no such restrictions on books after you legitimately
| purchased them for hundreds of years now.
|
| If you buy the hardcopy of the very same book, no such
| restriction would apply.
|
| Rip-off schemes, where the publishers uses DRM to cripple the
| reading experience by providing the content via a proprietary
| reader with bad UX, is one of the very reasons people resort to
| pirating in the first place.
| RomanPushkin wrote:
| What is ridiculous take? Authors want to sell access to
| information they've collected, improved, filtered, and so on.
| They invested tremendous amount of effort into making this
| available to you for a very low price (less than $1/month).
| The sharing is implemented the certain way, super
| inconvenient - but you have been informed multiple times in
| advance, and accepted that.
|
| > There were no such restrictions on books after you
| legitimately purchased them for hundreds of years now.
|
| For hundreds of years there were no book pirates, and
| computers. Even libraries buy books so they can share to a
| limited audience.
| pmyteh wrote:
| Book piracy is as old as copyright. There were presses
| putting out editions unauthorised by the Stationers'
| Company when they held a legal monopoly in England, and the
| US was a famous hotbed of book piracy throughout the 19th
| Century. Dickens, amongst others, was not happy.
| Simran-B wrote:
| > For hundreds of years there were no book pirates
|
| What did monks do if not copy books?
| _dain_ wrote:
| >If you're breaking the rules, authors won't get compensated
| and we won't see other Human Kenetics books. This should be
| illegal.
|
| so second-hand bookshops should be illegal too?
| ryanackley wrote:
| I'm sad this was downvoted. I think it would make a more
| interesting discussion for unpopular opinions to be present and
| engaged with rather than downvoted into invisibility.
|
| I think you make a fair point, if you don't want to agree to
| the terms of a transaction, don't participate in the
| transaction with the intent of breaking the terms. This is the
| implied covenant of good faith and fair dealing in any
| contract.
| thewebcount wrote:
| You seem to have missed the part where he paid for the book. He
| did pay for it, he just wanted to be able to use it in a more
| convenience way.
| JadeNB wrote:
| > Personally, I would love to pay hundreds for a limited access
| to books I would like to read.
|
| ... Why? I'd assume that this was sarcastic, except that the
| sarcastic reading seems to support the post with which you seem
| to be arguing, and the rest of the paragraph:
|
| > But folks aren't inspired too much about writing books today
| that can make them a living, since there are pirates
| everywhere. They want free content, because "information should
| be free". No, it shouldn't. Pay for it.
|
| seems to be sincere.
| thrdbndndn wrote:
| Almost all the non-major online ebook viewers are like that.
|
| Some of them would obfuscate the image resources, but that's
| about it.
|
| Big players typically have something "better". Bookwalker (a
| Japanese ebook vendor) has the most complicated viewers in my
| experience, and it always convert all the text to image on server
| side before serving. (And it obfuscates the images on server side
| too, so you have to reverse this process yourself if you want to
| download them).
| xvilka wrote:
| > it always convert all the text to image on server side before
| serving.
|
| It probably easy to recover to absolute precision with OpenAI-
| like models. I bet using AI could help a lot with recovering
| books in the future.
| innocenat wrote:
| BookWalker generally don't have exclusive rights to their
| books, so just buy from other place instead.
|
| On books that they do have exclusivity, though, people are
| doing exactly that, OCR the book and recreate the EPUB.
| thrdbndndn wrote:
| People also have already cracked their Android client so
| you can straight up get EPUBs.
|
| But it's not publically available since BW is very quick
| about patching their app once the crack is well known.
| chocolatkey wrote:
| Reverse-engineering Japanese ebook DRM is much more fun than
| English. You run into all sorts of mistaken cryptography
| assumptions, fun exploits, obfuscation etc. Hint for
| bookwalker: check out the app
| i_am_toaster wrote:
| I probably would have stopped at the pdf level because I'm lazy.
| Thanks for not being me, following through with your hatred, and
| doing a short write-up!
| frankfrankfrank wrote:
| He was clearly not happy with owning nothing.
| thot_experiment wrote:
| > More importantly, allow me to retire from having to write the
| tools that bypass your restrictions.
|
| FFS PLEASE! I waste so much of my life fighting DRM and anti-RE
| measures. It's so infinitely frustrating to fight against stuff
| that shouldn't exist in the first place. Someone wasted their
| time to waste my time and in the end we end up exactly in the
| same spot.
| chungy wrote:
| I'm flabbergasted by a digital-only edition being $82. That's a
| price I'd find step even for print. I paid less than that for the
| print+digital combination of The Art of Computer Programming
| Volume 4B.
|
| $82 and being forced into a time-limited web viewer. That's
| just... what's the market for that?
| BeetleB wrote:
| 20 years ago, most engineering textbooks were about $100 in the
| US. I discovered they sell them cheaper in Europe, and the cost
| to buy from Europe + shipping was still a lot cheaper.
| krapht wrote:
| Textbooks are a lot more work to prepare than people think, and
| the market is in the thousands. Especially specialist textbooks
| like this require the input of many upper-middle class
| professionals who aren't exactly cheap - much like software.
|
| I mean, how much time do you think it would cost to prepare a
| 400 page textbook on Amazon AWS, with curated examples and
| training exercises? How many man-hours of senior engineering
| talent would be required? How many copies could you
| realistically sell?
|
| Some textbook authors have previously posted on HN that the
| money is terrible in technical books, and the biggest benefit
| has just been credibility that the authors can use to sell
| themselves as consultants.
| ta988 wrote:
| The issue here is that publishers are raking up all the money
| and not even providing a good service anymore. The physical
| quality of my recent books, the formatting of the ebooks and
| the typesetting has gone downhill in the last 10 years or so.
| Books prices are still high, publisher profits are
| skyrocketing and quality of product goes down. It is time to
| setup a new way to support writters for publishing their
| books that doesn't involve those scammers. Not every writer
| can do their own typesetting and distribution.
| mdaniel wrote:
| > _Reverse engineering_ yet another eBook format
|
| > Surprise, surprise, our website was using one of the most
| popular ebook formats!
|
| :-/
___________________________________________________________________
(page generated 2022-12-25 23:00 UTC)