Learning Process case study: Making eBooks the Hacky Way

May 11, 2020 - By

I’ve never shared an early, messy workflow with anyone before, but I’ve been thinking a lot lately about different learning styles, and this is a good case study of my iterative approach. I do my best learning in-the-moment, and on real projects.

My current eBook workflow is anything but elegant, but is less than a week old. This is the exact opposite of being a scholarly perfectionist who learns a craft through deep understanding; it’s the Cowboy “do it live” approach. I shudder at how much I don’t know, and how hacky this workflow is – but it works. I’m a bit proud of this, since I got this tricky work done fast, and because I figured it out way past my bedtime, with a project that launched immediately thereafter. There was no planning ahead. The author Jason said “Hey what if…” and then we just did it!

Serendipitously, I had experimented for an hour on a Bible Translation HTML-to-eBook build in the week before (work meriting its own future post). At the point where I started lending a hand with this book I could say “I know what I don’t know”, particularly how unintuitive and important the TOC (table of contents) formatting is, and how even a TOC that works won’t necessarily appear in a Kindles “Go To” menu. I had also researched eBook publishing enough to see that the topic is saturated on search with lots of scummy “get rich quick” style posts. Converting the document into some other proprietary platform-specific format would be its own battle which I wanted to avoid.

I’ve been building websites for 25 years, and I can see now that there are subtle skills in how to use search engines to find solutions to specific problems. I am way more adept at this with web than eBook publishing. I would have thought the overlap on these disciplines was a lot broader.

The Workflow

The master copy was in a Microsoft Word doc. First, Jason got the master into a state that would be the best source for conversion. We published that to PDF, cinchy (well, sort of; the Mac print-to-PDF broke the links for some reason, we had to export from within Word). Some trial and error showed us that the page-break character was creating havoc on eBooks, Jason solved this on his end. We also learned the hard way that the way white space was added around headings would create other “level 1” (aka h1) “(Untitled)” items in the TOC which had to be removed.

Once the master and PDF were sorted we were able to make a mobi and epub with the following workflow. I am surprised how much we used an online file-converter, I typically only use online conversion tools in a pinch, we landed on this after some trial and error from my previous experiments. Last week’s experiments with Pandoc (a command line tool) were fruitful, but results through Calibre were better. Where these failed the Convertio webapp came through.

  1. https://convertio.co: Drag a version of the Docx (without cover image), convert to to epub.
  2. Calibre to convert new epub to mobi
    1. I set the cover image here
    2. Don’t select the option add blank pages before h1 (as I had to do in a my previous document), this adds extra completely blank page
    3. Don’t make an epub in this manner, for some reason forces text color to black, even in dark mode’. I bet there’s a setting in Calibre, but the docs are pretty long, often old, and I’m trying to execute in the least time possible… my gut says outside-the-box experiments will be faster
  3. Upload the new Calibre-Mobi into Convertio, convert to epub – fixes the font color: it will be white in dark-mode, black in light-mode.
  4. Finally, Verification based on finding the most common broken issues:
    1. Does front TOC work (this was the link set made in Word)? No “unknowns” line items
    2. Does back TOC work (epub and mobi generate TOC at the end of the doc)? Not sure this is needed, but as a web designer it feels like footer-links and giving the reader somewhere to go at the end
    3. Right click on mobi > Open in Kindle for Mac: Does Kindle TOC work? If that looks good I’d email the mobi to my physical Kindle to test on a device too, particularly keeping an eye on page margins
    4. Look at epub in Calibre viewer in dark mode – text ok?
    5. Paragraph spacing ok? Number sizing and sequencing ok? This document had a large ordered list, early experiments had the numbers all be “1”.

A bit more hoop-jumping than I like, but compiling always feels a bit like a black box.

Fiddle and fuss with tools, design, develop, QA, repeat. Big-picture the workflow I’m leaning into here is like web development.

Weird Issues

  • I was initially very surprised about the persnickety TOC – I’m not sure what conversion is happening; this is where I hope to get more academic understanding. I would have thought a <ul> with links could be set as the document’s primary TOC. Maybe it can, but the relationship between a web document and eBook formats isn’t what I had assumed. I really wanted to author a perfect HTML version and have a real eBook ready TOC in the markup.
  • Right Clicking on a mobi to “Open in Kindle” for Mac only works if that app is already open, otherwise the loading screen hangs. Kindle Mac app also will sometimes freak out that the file already exists… but not every time. So you have to delete previous versions, one at a time, tediously, with right click menus.
  • In my previous experiments HTML to mobi worked while HTML to print-to-PDF to mobi broke links.
  • I predict that a less simple document would have had challenges. Quote formatting, media, footnotes/endnotes/bibliography–there are some battles left to wage I’m sure.
  • I am a morning person, doing stuff like this in a single sitting past 8pm was a fun challenge, one I should keep rare.

Conclusion / Iteration Time!

“Messy, but it works” was definitely the right approach for this project, but not forever. My learning process is iterative; and on the next process I hope to find out how to do more of this within Calibre, Pandoc, or whatever the ideal tool is, and hopefully not an online converter. Ideally I won’t wade through more of the scummy google results and will find an industry expert who has experience in proper ways of publishing these eBook formats.

Jason’s Book is live (and free) at the Operation World blog for CoVid-19: http://covid-19.operationworld.org/global-transmission-global-mission

Wee Updates

Standard eBooks CLI: Standard eBooks looks like a super promising next step for research. As always, learning to be continued.

Apple Pages: I learned that some people author their epub files from Apple Pages. It has a nice export, but if I read the epub in Calibre with dark-mode the text stays black, like in my earlier trials. This makes me wonder if it’s a non-issue on actual devices. Am I just picky? Still weird.

Categorized in:

This post was written by ArleyM