Learning Process case study: Making eBooks the Hacky Way

May 11, 2020 - By

I’ve never shared an early, messy workflow with anyone before, but I’ve been thinking a lot lately about different learning styles, and this is a good case study of my iterative approach. I do my best learning in-the-moment, and on real projects.

My current eBook workflow is anything but elegant, but is less than a week old. This is the exact opposite of being a scholarly perfectionist who learns a craft through deep understanding; it’s the Cowboy “do it live” approach. I shudder at how much I don’t know, and how hacky this workflow is – but it works. I’m a bit proud of this, since I got this tricky work done fast, and because I figured it out way past my bedtime, with a project that launched immediately thereafter. There was no planning ahead. The author Jason said “Hey what if…” and then we just did it!

Serendipitously, I had experimented for an hour the week before on a Bible Translation HTML-to-eBook build in the week before (work meriting its own future post). At the point where I started lending a hand with this book I could say “I know what I don’t know”, particularly how unintuitive and important the TOC (table of contents) formatting is, and how even a TOC that works won’t necessarily appear in a Kindles “Go To” menu. I had also researched eBook publishing enough to see that the topic is saturated on search with lots of scummy “get rich quick” style posts. Converting the document into some other proprietary format would be its own battle which I wanted to avoid.

I’ve been building websites for 25 years, and I can see now that there are subtle skills in how to use search engines to find solutions to specific problems. I am way more adept at this with web than eBook. I would have thought the overlap on these disciplines was a lot broader.

The Workflow

The master copy was in a Microsoft Word doc. First, Jason got the master into a state that would be the best source for conversion. We published that to PDF, cinchy (well, sort of; the Mac print-to-PDF broke the links for some reason, we had to export from within Word). Some trial and error showed us that the page-break character was creating havoc on eBooks, Jason solved this on his end. We also learned the hard way that the way white space was added around headings would create other “level 1” (aka h1) “(Untitled)” items in the TOC which had to be removed.

Once the master and PDF were sorted we were able to make a mobi and epub with the following workflow. I am surprised how much we used an online file-converter, I typically only use online conversion tools in a pinch, we landed on this after some trial and error from my previous experiments. Last week’s experiments with Pandoc (a command line tool) were fruitful, but results through Calibre were better. Where these failed the Convertio webapp came through.

  1. https://convertio.co: Drag a version of the Docx (without cover image), convert to to epub.
  2. Calibre to convert new epub to mobi
    1. I set the cover image here
    2. Don’t select the option add blank pages before h1 (as I had to do in a my previous document), this adds extra completely blank page
    3. Don’t make an epub in this manner, for some reason forces text color to black, even in dark mode’. I bet there’s a setting in Calibre, but the docs are pretty long, often old, and I’m trying to execute in the least time possible… my gut says outside-the-box experiments will be faster
  3. Upload the new Calibre-Mobi into Convertio, convert to epub – fixes the font color.
  4. Finally, Verification based on finding the most common broken issues:
    1. Does front TOC work (this was the link set made in Word)? No “unknowns” line items
    2. Does back TOC work (epub and mobi generate TOC at the end of the doc)
    3. Right click on mobi > Open in Kindle for Mac: Does Kindle TOC work? If that looks good I’d email the mobi to my physical Kindle to test too
    4. Look at epub in Calibre viewer in dark mode – text ok?
    5. Number sizing / paragraph spacing ok?

A bit more hoop-jumping than I like, but compiling always feels a bit like a black box.

Fiddle and fuss with tools, design, develop, QA, repeat. Big-picture the workflow I’m leaning into here is like web development.

Weird Issues

  • I was initially very surprised about the persnickety TOC – I’m not sure what conversion is happening; this is where I hope to get more academic understanding. I would have thought a <ul> with links could be set as the document’s primary TOC. Maybe it can, but the relationship between a web document and eBook formats isn’t what I had assumed.
  • Right Clicking on a mobi to “Open in Kindle” for Mac only works if that app is open, otherwise the loading screen hangs. Kindle Mac app also will sometimes freak out that the file already exists – but not every time. So you have to delete previous versions, one at a time with right click menus
  • In my previous experiments HTML to mobi worked while HTML to print-to-PDF to mobi broke links.
  • I predict that a less simple document would have had challenges. Quote formatting, media, footnotes–there are some battles left to wage I’m sure.
  • I am a morning person, doing stuff like this in a single sitting past 8pm was a fun challenge.

Conclusion / Iteration Time!

“Messy, but it works” was definitely the right approach for this case, but not forever. My learning process is iterative; and on the next process I hope to find out how to do more of this within Calibre, Pandoc, or whatever the ideal tool is. Ideally I won’t wade through more of the scummy google results and will find an industry expert who has experience in proper ways of publishing these eBook formats.

Jason’s Book is live (and free) at the Operation World blog for CoVid-19: http://covid-19.operationworld.org/global-transmission-global-mission

Wee Updates

Standard eBooks CLI: Standard eBooks looks like a super promising next step for research. As always, learning to be continued.

Apple Pages: I learned that some people author their epub files from Apple Pages. It has a nice export, but if I read the epub in Calibre with dark mode the text stays black, like in my earlier trials. This makes me wonder if it’s a non-issue on actual devices. Still weird.

Categorized in:

This post was written by ArleyM