recentpopularlog in


« earlier   
Highlights - Sawyer Hollenshead

"How I’m exporting my highlights from the grasps of iBooks and Kindle
Using email, AWS, Siteleaf, and GitHub

A few years ago there was a little startup called Readmill that gave a glimpse at what an open, independent reading platform could look like. You could import your books into their beautiful reading app and highlight text as you read. Your highlights would sync with your Readmill account and other people could follow along to see what you were highlighting (and vice-versa). I discovered a bunch of new books and met some new faces this way. I even built a product that tied in with their API. Then Readmill got acquired by Dropbox. The open, independent reading platform was no longer open or independent, and shutdown in July 2014. Since their shutdown, the state of digital reading platforms has been pretty sad.

Now, my reading takes place in a train on my phone (iBooks) or in sunny Prospect Park on my Kindle. I still highlight as I read, but they don’t sync anywhere. They’re typically scattered between two walled gardens, and 99% of the time I don’t come back to reflect on what I’ve highlighted. I might as well be posting screenshots of the text to Twitter like a buffoon (✋guilty).

So after stewing in frustration for quite awhile about the current state of digital reading platforms, I decided to do what any sane programmer would do: Devise an overly complex solution on AWS for a seemingly simple problem (that two companies with a combined market cap of close to a trillion fucking dollars can’t be bothered to solve).

The ultimate product was (Skip to the bottom for links to the code).

The problem: How do I gather all of my highlights from iBooks and Kindle and put them into one collection, preferably online, where I can share, browse, and reflect on everything I’ve read?
The solution: Email.

Actually, it’s a bit more complicated than just “Email”. Yes, I suppose I could just email the highlights to myself and be done with it. Now that I think about it, maybe I should have started there. But I didn’t, I jumped right to this: I created a new email address (eg. and hooked it up to Amazon Simple Email Service (SES). Using SES, my email address receives email I send to it and stores the email as essentially a text file in Amazon S3 (aka an online folder that stores files). Amazon S3 is smart though and can notify other services when a new file is added to it. So I setup my S3 folder to notify another Amazon service, Amazon Lambda, whenever a new email is received. Lambda is the “brains” of this whole flow. It’s given an input, the email S3 just stored, and runs code on that input."

Sending and parsing highlight emails

The code that I setup Lambda to run does a few things: First, it reads the email and identifies the source of the highlights as either iBooks or Kindle. Emails with iBooks highlights contain the highlights in the body and Kindle highlights are sent as attachments. Why?

iBooks provides a fairly nice user experience for emailing your highlights, so all I have to do is select the highlights I want to share and email them to my address.

Kindle is a bit more of a monster. For books that I’ve purchased through Amazon, my highlights get synced to the Kindle highlights page, possibly one of Amazon’s most neglected pages. Using a bookmarklet, I export all these highlights as a JSON file. Next, I email the JSON file as an attachment to my SES address.

Publishing the highlights online

Now that my Lambda code knows the source of the highlights, it parses the highlights from the email and we proceed on to the next step: Saving the highlights to Siteleaf. Siteleaf is a content management system that myself and the team at Oak have been working on. Siteleaf allows you to manage your website’s content in the cloud and then publish your site as static HTML to a web host of your choice. Siteleaf also has an API, which I’m using to save my highlights. Once my highlights are saved to Siteleaf, Siteleaf automatically syncs the new highlights to GitHub as Markdown files. At this point, my highlights are saved to Siteleaf and accessible through the CMS and API. They’re also saved as Markdown files in a GitHub repo. Pretty cool. With one more click in Siteleaf, I then publish these highlights to my website, hosted on GitHub Pages. Now they’re also saved as HTML pages and accessible to everyone online. Even cooler.

(Note: The Siteleaf functionality mentioned above is currently in beta and not yet open to everyone. You can apply for access though — I know a guy.)

The irony that I’m using all of these Amazon services to solve a problem that Amazon itself is a part of isn’t lost on me. Like I said at the beginning, this is an overly complex solution to a problem that seems so simple — but it works for me. Now that I have all the pipes connected, when I finish reading a book, I send one email and my highlights are ready to be published to my site. Whether or not Apple, Amazon, or some other company ever makes browsing your ebook highlights and notes easier, I hope to always have a method of my own. If you have your own workflow, I’d love to hear about it.

View the code on GitHub
Instructions and code for the Lambda function can be found on GitHub. Additionally, the code for is also available GitHub."]
via:caseygollan  sawyerhollenshead  books  libraries  digital  digitallibraries  readmill  kindle  ibooks  webdev  siteleaf  html  annotation  webdesign 
march 2016 by robertogreco
Readmill Epilogue - Readmill
Readmill was an ebook reader for iOS and Android.
readmill  honesty 
april 2014 by stephenlk
FBReader is Adding Support for DRM-ed eBooks - But Not For the Reasons You Think - The Digital Reader
新的 FBReader Android App,將加入 ExpressPlay DRM,一個更輕簡且便宜的保護方案,在Adobe 與 DRM-Free 兩個極端之間,走出一條生存之道。
readmill  from twitter
april 2014 by wastemobile
Readmill to Close
Readmill, a social reading app and company based out of Germany, is closing. TechCrunch first reported the news that Dropbox has acquired the app, with plans to shut it down. TechCrunch reported that Readmill's staff would be moving to San Francisco to work for Dropbox, but could not confirm details from Readmill's CEO, Henrik Berggren. In a letter posted to its users, Readmill said its "story ends here" and that it " failed to create a sustainable platform for reading" despite its belief that the reading experience was "meant to be shared." (...) The shuttering of Readmill, for many in the book industry, brings into question the viability of social reading itself, which has been one of a handful of areas some book-focused startup's have explored. Whether social reading is a technology users want--and a viable platform for earning revenue--is now a larger question.
Readmill  Berggren  Dropbox  annotation  social-reading 
march 2014 by sentinelle
Readmill Epilogue - Readmill
I really enjoyed Readmill. That and Marvin are easily the best two ePub reading experiences in their own unique ways. I'll Miss Readmill and it's beautiful UI and social component. I'm sad to see it go, but maybe something grander will come of it.
readmill  ebook  ebooks  apps 
march 2014 by cbucher1
Readmill Epilogue - Readmill
"Readmill’s story ends here. Many challenges in the world of ebooks remain unsolved, and we failed to create a sustainable platform for reading. For this, we’re deeply sorry. We considered every option before making the difficult decision to end the product that brought us together."
Reading  Readmill  Shutdown  Article  Business 
march 2014 by jsm
User data export - Readmilling
"Readmilling is a collection of scripts for use with Readmill's API. Created by Morbus Iff.
These scripts are not affiliated with or created by the fine chaps and chapettes at

User data export
You put it in, now get it out.

export user dataBe aware: If we've not seen this user before, or its data has expired, it might take a few minutes before you'll get results."
readmill  export  data  via:maxfenton 
march 2014 by robertogreco

Copy this bookmark:

to read