published Oct 21, 2013
Today we just deployed a new feature: the Write tab. It’s hidden under the Actions tab.
Yeah, we’re going to reorganize the author app so things are less nested, but that’s another blog post for another time…
Anyway, back to the Write tab…
For many of you, this will be totally unimportant. We’re still 100% committed to the notion of “write on your own computer (or tablet), using your favourite tools”, etc. And we still think that Dropbox is a fantastic way to sync files between your computer and Leanpub.
However, that said, there are times when it’s nice to be able to edit a book in a web browser. For example…
- If you’re on a shared computer and can’t use Dropbox and local files, etc.
- If you just want to make a really quick fix or change to your book, and you’re already in your browser.
- If you’re on your tablet or phone.
- If you want to show a friend how great Leanpub is, and wish you could get from the create account to book preview exists status in 3 minutes, instead of spending the time signing them up for Dropbox, accepting share requests, etc.
Ironically, I (Peter Armstrong) learned the lesson of #1 firsthand recently: I was doing a 2 hour, hands-on “how to use Leanpub” workshop in Brisbane for a room full of fiction authors at GenreCon. Problem is, the computers in the lab could not save local files. Epic Workshop Fail – but also, a great Customer Development experience.
There should be absolutely no reason that anyone in the world can’t sign up for Leanpub and create a book in 5 minutes. None.
So, because of this epic fail, we now have a Write tab, and we hope it’s useful.
Right now, it’s very much a Minimum Viable Product. You can’t even upload images or a title page.
To try it out, go to https://leanpub.com/YOUR_BOOK/write.
Here’s what works right now:
- You can load up the page and see a list of the files in your book.
- You can edit the files and then save them.
- Saving the files syncs the changes to Dropbox and does a git commit internally in Leanpub
- You can edit the files in your sample (by using checkboxes to choose what is in the sample).
- You can reorder files (but see point 1 below).
- You can create and delete files.
- You can start a preview.
- You can download your PDF, EPUB and MOBI files from the Write tab.
Here’s what we’re planning to fix soon:
- There is no notion of “this file has changed and you are going to lose stuff if you move away from it”.
- The display of the preview progress bar is a little flaky.
- You can drag and drop to reorder files, but there’s no indication that this is possible.
- There is no image tab.
- There is no Markdown help tab.
Here’s the most obvious two enhancements to consider:
- a nice distraction-free Markdown editing environment
- a WYSIWYG editor that degrades gracefully
Here’s the most important thing:
Using the Write tab is not a decision you need to make up front.
The Write tab just lets you edit Markdown. It’s the dumbest UI imaginable: it took about 15 minutes to wireframe in Keynote in a hangout with Scott.
It’s not a decision that an author makes about whether they want to edit the book in their browser using some WYSIWYG editor or whether they want to edit the book in Dropbox. Our intention here is for people to sign up for Leanpub, kick the tires using the Write tab, generate a preview, download the previewed book, realize the potential of what Leanpub offers – and then accept the Dropbox share request and edit their books using their favourite tools…
We’re never going to do a visual editor that precludes us from supporting Markdown editing in Dropbox. We may, however, if this takes off, do a visual editor that degrades gracefully to editing Markdown text if it encounters something that it does not understand. However, that will be non-trivial to build, so doing a nicer looking distraction-free Markdown editor would presumably be a higher priority. If and when we look at doing a visual editor, we will consider all the available starting points, including the recently released Sir Trevor, etc.
Anyway, thanks for reading. We hope the Write tab is helpful, and not just for demos…
P.S. No, we did not do this because I’m speaking at Books in Browsers on Friday, Oct. 25 at 10:30 AM. But, the timing is a nice bonus, and I’m very happy (thanks Scott!!!) that it got done beforehand. (For any Leanpub authors in San Francisco, let Leanpub buy you drinks: October 25, 6 PM, the Kanpai Lounge in Hotel Nikko, 222 Mason Street…)
Lean Publishing 2013 "World Tour" - let Leanpub buy you a beer in Brisbane (Oct. 12, 5 PM, Fifth Element) or San Francisco (Oct. 25, 6 PM, Hotel Nikko)
published Sep 29, 2013
Update October 8:
Here are the dates and times:
Brisbane: October 12 at 5 PM at Fifth Element, 188 Grey Street
San Francisco: October 25 at 6 PM at the Kanpai Lounge in Hotel Nikko, 222 Mason Street
(Frankfurt on October 7 got cancelled.)
All I need is a fake rock concert T-shirt…
I almost never go to conferences, and I only go when I’m speaking, for reasons outlined best in 2010 by Mark Suster… However, in February I did a talk about Lean Publishing and Leanpub at TOC2013 in New York. It was well-received, and had meaningful benefits for Leanpub, including an article in The New Yorker blog. And since I got it recorded it keeps providing benefits to Leanpub.
So, this October, I’m going to try an experiment and do a round-the-world flight that will include three conferences. I’d like to buy Leanpub authors in those cities a beer (or a coffee, or a glass of wine) and learn about their Leanpub experience and how we can improve it…
1. CONTEC, Frankfurt (Conference = October 8, Possible Leanpub Drinkup = October 7)
Kat Meyer is involved in producing this event, and she is genuinely great, so this should be a great conference. So, if anyone here is in the Frankfurt area, go to the conference!
I have a 20% discount code that the organizers want me to share: CONTEC13SP20
The registration form can be found at: http://www.buchmesse.de/en/academy/programme/00339
I’m flying out right after the panel on October 8 that I’m on (it turns out Australia is far away from Germany, and there are lots of time zones ;), but if anyone is in the Frankfurt area on October 7, email me. I’m sure there is beer in Germany in October…
2. GenreCon, Brisbane (Conference = October 11-13, Possible Leanpub Drinkup = October 12)
I’m doing a short talk and a how to use Leanpub workshop. I think the talk is 20 minutes and the workshop is 2 hours.
I assume that on October 11th I’ll be a zombie from the jet lag, but if there are any Leanpub authors in Brisbane, ping me on October 12!
3. Books in Browsers, San Francisco (Conference = October 24-25, Possible Leanpub Drinkup = October 25)
I’m talking for 20 minutes on the morning of October 25. Also, John Maxwell (an SFU prof who is a huge friend of Leanpub) is talking on the afternoon on October 25.
If there are Leanpub authors in the San Francisco on October 25, ping me. The conference should end at around 5 PM on the 25th, and I’m free after…
published Sep 19, 2013
This interview was recorded on September 3, 2013.
Len Epp: I’m here with Taylor Otwell, creator of the Laravel PHP framework, and a Software Engineer at UserScape. Since it was created in 2011, Laravel has become one of the most popular open-source PHP projects in the world. The framework has a large, growing, and devoted community, with thousands of developers using it to build web applications.
Taylor is the author of the Leanpub book, Laravel: From Apprentice To Artisan: Advanced Architecture With Laravel 4. In this interview, we’re going to talk about Taylor’s career, and the creation and development of the Laravel framework. We’ll also talk about his book, his experiences using Leanpub, and ways we can improve Leanpub for him and other technical authors.
So, thank you Taylor for being on the Lean Publishing Podcast.
Taylor Otwell: Thanks for having me. I’m excited to be here.
E: Thanks! I’d like to start by asking you for a brief introduction to your career - how did you get interested in programming, and what led you up to the point where you created the Laravel framework?
O: I’ve been interested in computers ever since I was a kid, like probably many of us were. I went to Arkansas Tech University here in Arkansas, in the United States, and majored in Information Technology. In college, I learned kind of the basics, C++, you know, but I wasn’t really familiar with open-source programming or programming as a hobby, or hacking, you know, and when I graduated from college, I went to work for a large trucking company here in the US called ABF Freight. There, I started doing mainframe COBOL and .net at the same time, which was kind of an odd mix. They actually had a six-month training program, taught me .net. One of the great things about working with ABF, was I worked with a very good group of programmers. They taught me a lot about programming, a lot about software design, and started to introduce me to open-source software. So now, I was doing .net at work, but in my free time I had ideas for side business, or whatever, things I thought I’d like to start. I knew from my college experience that PHP was a popular web language, that was cheap to host, and so I thought, Hey, why not? I’ll learn some PHP and hack on some of these ideas at night. It’s cheap to host, I can just throw the files up on a server, and I’m good to go. So, that’s how I started hacking and working on PHP.
Now as far as the Laravel framework, I was using another framework called CodeIgniter, I was continually hacking the core of that to make it behave more how I thought it should behave, from my experience with ASP.NET. I wanted things like dependency injection, and a better templating system, a better ORM, and so I started hacking on Laravel in my free time at night, and released it after about five or six months of working on it. It was a very small framework at first, but of course now it’s grown to be quite popular, and a full stack framework.
E: Did you work on it independently, or did you have people helping you along the way for the first release?
O: I worked on it totally independently. I did all the coding, and of course drew inspiration from other projects, so indirectly, there was lots of help from projects that were built before mine. But yeah, I was the only one doing the coding.
E: Was there a particular type of developer that you had in mind when you were building it? Is there a particular type of person that it clicks with?
O: It seems to click a lot with people who want to build something very rapidly. People who like lots of great documentation. From the very first day that I released Laravel, I had total and complete documentation over the project, because I knew that documentation was king for new open-source projects. If a project was thrown out there with not very thorough documentation, like a lot of projects are, it’s a lot harder for it to gain steam. So I knew that even if the framework wasn’t the best PHP framework out there on the first day of its release, if it had good documentation, that I could build a solid community around that, because people would be a lot more inclined to use it if they could learn more about it from the docs.
E: It must have been very exciting to see the community develop around it. Did that happen quickly, or did that take some time, and did you have to do a lot of encouraging?
O: It took a little bit of time. Building a community nowadays is a little bit easier, thanks to things like Hacker News and reddit - you can gain some early adopters pretty fast. So, early on we probably had a few dozen people that were regularly using it, and that quickly ballooned, and a lot of that because I was very persistent in doing screencasts, new features. Laravel at first was kind of half-framework, half-marketing effort by me, so I was constantly trying to put new information out there, advertise new features on various PHP websites, and keep getting the word out, even after the initial launch.
E: You’ve recently, just earlier this year, released version four. Can you tell me a little about what’s new in this version?
O: With version four, we wanted to make a big architectural change under the hood, because versions one through three were kind of, we were still feeling our way toward how the framework should behave, and how we wanted it to look. With version four, we knew we needed to improve on a few things. One was testability, we needed the framework to be very testable out of the box. We wanted good PHPUnit support out of the box. So that was a big focus. We also wanted to start playing nicely with Composer, and we adopted Composer, which is a PHP package, kind of installation system, sory of like Bundler for Ruby. We adopted that as our primary way of distributing packages. So that opened up a whole ecosystem of packages that are easily installable and integrate really well with Laravel. I think that’s going to be very beneficial for years to come.
E: Were your decisions about new functionality and features driven by signals you were getting from your community?
O: Yeah, they were. I’ve always been really involved with the community personally. Early on, in the forums, and now more with IRC and Twitter, but I’m always listening to community feedback and seeing how things are trending. Plus I work on a large Laravel application day-to-day myself. So, I tend to identify the awkward spots in the framework fairly quickly too, because I’m using it so much, in a real world setting. The community drives the development in a lot of ways, and I add my own insights here and there too.
E: I’ve seen people refer to the community as a “family” online, it seems very friendly and supportive.
O: That was one of the goals from the start: to have a very inclusive community, but at the same time not a stagnant community. We want to be very welcoming of newcomers, so to speak, but we don’t want to dumb things down too much. We want to encourage people to grow, and that’s one of the things people tell about Laravel, is it sort of grows with you, in the sense that, when you first start using it, it’s very easy to build out an application, and to get some basic routes going, to log people in, to put some stuff in the database. That’s all very quick and easy. But at the same time, it grows with you, and as your architectural skills grow, you have access to more powerful features, like the IoC container, like background queueing, like doing some Redis caching, and more powerful features. So the framework sort of evolves as you evolve as a developer, which is kind of nice.
E: That’s fantastic. I know you’ve just returned from Laracon 2013, which was a two-day Laravel conference based in Amsterdam. I think that was the second Laravel conference that’s happened, and the second this year. Can you tell me a little bit about how it went, and what the highlights were for you?
O: Yeah. So, like you mentioned, this was the second Laracon. We’ve having an annual Laracon US, and an annual Laracon EU right now. The first one was in Washington, DC, and the second was in Amsterdam. I think it went really well. We actually had Fabien Potencier - I’m sure I’m not pronouncing that right! - he is kind of the maintainer of the Symfony framework, which we use, so we consume about seven or eight of their components. We also had Jordi from the Composer, kind of co-maintainer, so that was really great, because we had some prominent members of the PHP community that don’t necessarily even use Laravel on a day-to-day basis, but have had a big impact on Laravel. I think people enjoyed meeting them, and talking to them, and bouncing ideas off them. It was just a fun time of hanging out. You know, conferences are always kind of like, half-learning, half-social, or maybe even like 60% or 70% social, 30% learning. So yeah, it was fun to hang out with everybody, meet people you’ve talked to online, and stuff like that.
E: I was going to say, it’s must be very interesting, I mean it’s probably been a roller coaster couple of years to go from just launching something you built on your own, and now you’re going to a conference in Europe and meeting, I would imagine, a few hundred people who are using it.
O: Yeah, it’s really crazy. It’s just kind of wild. It’s weird, to go somewhere, and have people say they’re using Laravel. It seems like not too long ago I was sitting in my office building it. It was fun to meet people that I’ve known for a couple years now.
E: You mentioned your day-to-day work at UserScape. Can you explain a little bit about what UserScape is doing?
O: Yeah. At UserScape we have two products, we have HelpSpot, which is an on-premise help desk solution. It’s very powerful, very flexible. And you actually can download it and install it on your own servers, which is nice, a lot of companies really like that. Our second product is Snappy, which is also a help desk solution, but it’s a little lighter-weight. It’s a SaaS application, so you actually just log right in, over the internet, you don’t need to download or install anything. It’s working out really great, and it’s got a great interface, live-updating with your emails, and you can assign stuff to other people, using @ tags. You can hashtag emails to categorize them, so it’s a really cool app - it’s besnappy.com if you want to check it out.
E: I’m sure some of our listeners would be interested to know how you manage being the architect of Laravel, with your work at UserScape, day to day.
O: I have kind of a routing schedule, where 8:00-5:00 I work from home, I work on UserScape stuff, that’s primarily Snappy, which is built on Laravel. And then, after work, from like 5:00 to 9:30 or 10:00, I just hang out with my family, and then after my wife goes to bed, I work on Laravel for a few hours, from say 10:00 to midnight, or 10:00 to 1:00. And then get up the next morning and repeat. Sometimes I’m too tired to stay up and work on Laravel, but usually a few days a week I get to work on it at night. And Ian, my boss at UserScape, also gives me the last week of every month to work on Laravel, when I usually go on a pull request rampage, and close like hundreds of issues. So that’s really nice.
E: That’s great to have that kind of support. I know you just released version four, but can you give me some indications about what you see in the future for Laravel, say next year, or even the year after that?
O: One of the complaints we had about Laravel early on was that we released versions really quickly. At first, that wasn’t really am issue, because we were all hacker, early-adopter types, so it was like, we don’t care if you break backwards compatibility, just give us cool new stuff. But as the framework has grown, and more businesses are using it, we had to kind of play it a little more mature, in terms of our releases. So we’ve adopted a six month release cycle, which started in this May. So that means Laravel 4.1 will be in November, and Laravel 4.2 will be in May of 2014. Right now, there’s no really backward-breaking being planned for those releases, but we are releasing some exciting new features. We’ve got some new components in the framework to let you make deployments easier. We’ve got some new debugging capabilities coming out for the release in November. So, we”ll have two point-releases, they’ll be incremental releases, adding new stuff but no major rearchitecting or anything.
E: I’ve heard you speak in a couple of podcasts at Laravel.io about the development of PHP in the last few years and how a lot of people who may make negative comments about PHP in forums often don’t even know where PHP has gone. Can you say a little bit about that, about how PHP has changed in the last couple of years?
O: Yeah. I tell people that when I first started writing Laravel, and using PHP a few years ago, I almost felt guilty for not using something else. You know what I mean? I almost felt guilty for not using Rails, I felt like less of a programmer. But now that PHP has grown so much, like 5.3, we’ve got like namespaces and closures, and now with 5.4 and 5.5 - I’ve talked to a lot of other people, and they felt the same way, you know, they felt like, Argh, I really shouldn’t be using PHP, I should be using Ruby or Python, or something like that.
But now I think with these new tools, they feel comfortable with PHP, and they don’t feel bad for using it. I never think anymore, personally, like, Oh, I should go try Rails, because everything that I need is pretty much available in PHP, and Laravel, feature for feature-wise, is really getting to the point of being on par with Rails. Especially with Laravel version 4. I think it’s a really exciting time for PHP, and tools like Composer, and this emphasis on growing this package ecosystem for PHP is really nice. Because, when I do look at Ruby and Rails, one thing that I do envy is the sheer amount of cool gems and stuff they have. As Composer grows in popularity, and more and more people build these packages, hopefully we can catch up to that. That is one area they’ve been whooping up on PHP. But I think we’re heading in the right direction for sure.
E: I’m curious, you say when you were first launching it was really important to build community and have documentation and screencasts and things like that. I’m wondering, if media attention is something that you’re interested in, or it something that just kind of drops out of all the other activities you do? Or is it something you completely ignore?
O: I try to do, still market the framework in a sense. I try to tell people I view the framework as a product. I don’t look at it as just a side project anymore. I want Laravel to be - I want it to have a certain personality and presentation, and so I spend a lot of time thinking about that, thinking about how the docs look, how they read, how the framework feels, in terms of its API - just very, almost obsessive in terms of how the framework comes across, and just the way that it presents itself - I want it to present itself as kind of a light, accessible tool, that’s out of your way, that’s easy to use, and helps you build a lot of great stuff really rapidly.
I don’t know if that made sense!
E: It does, and it meshes with what you said earlier about being friendly to new users, and generally having an inviting atmosphere.
O: Definitely. And a lot of this, I mean, everyone’s inspired by what Apple has done, in the sense that they’ve made technology very accessible and usable, things like the iPhone. And I kind of think of Laravel as a product, obviously nowhere near the scale that Apple operates on, but I wanted to have that same kind of accessibility.
E: I know you get asked this question a lot, but I have to ask it on this podcast, because some people might not know - can you briefly explain where the name “Laravel” comes from and how you chose it, or made it up?
O: The name actually has no meaning. I was struggling to come up with a name, and I just started thinking of words that sounded cool in my head. It does rhyme with a word that has meaning. In the kids’ books, The Chronicles of Narnia, there’s a place called Cair Paravel, which is where all the kings and queens live, and so it kind of rhymed with that, Laravel and Paravel, so it just kind of stuck. So, it has no meaning, it just kind of stuck in my head. I was having a hard time coming up with names, so I just kind of gave up coming up with meaningful names!
E: I was just struck by how good it is, on all sorts of levels - naming is something people talk about a lot on the web, and finding something with seven letters, that was, I’m sure, available, and is spellable, is really important, and that name captures all of those things.
If you don’t mind, I’d like to move on, to asking you some questions about your experience with Leanpub and the Lean Publishing process. Can you tell me how you found out about us, and why you chose us for your publishing platform?
O: Yeah. Actually I found out about Leanpub from Dayle Rees, who’s like, maybe the all-time highest grossing author on Leanpub. He also wrote a Laravel book, kind of a more introductory book, and he really enjoyed Leanpub, and recommended that I try it out.
E: It’s been an extraordinary thing for us - I’m just checking our bestsellers right now, our all-time earnings, and Dayle’s “Code Bright” is on top, and yours is number three, and Dayle’s first book is number seven. And then, Jeffrey Way’s book, Laravel Testing Decoded, is the tenth, so of our top ten, three of them are Laravel books.
O: Yeah, it’s wild. It’s really crazy. But yeah, he’s the one that turned me on to Leanpub as a platform.
E: And how long has Dayle been involved with the project?
O: Gosh, let’s see. It’s probably been a year and a half now, I’d say. He’s been around for quite a while. But, with the latest version of Laravel, he really came on board and started helping me manage the pull requests, because we get quite a few every day, and has been really helpful on IRC, answering questions and stuff.
E: I have just a general question. How would you describe your experience using Leanpub to write and publish this book?
O: Leanpub is really awesome, because I kind of feel like I don’t notice Leanpub, and I mean that in a good way. I just write my Markdown, and go to Leanpub and hit publish, and you know, you all handle everything. I never have any Leanpub headaches. It’s just really nice because Markdown is such an easy way to write, and because all the Laravel documentation is already in Markdown, I’m kind of used to writing Markdown. It’s just such a simple little markup for writing. And just the way that Leanpub lets you manage everything very easily, it’s just so straightforward, and I could get started really fast. Even now, with multiple translations of the book, it’s very easy for me to set up a new book, and get a coauthor set up, and divide out the royalties, you know, it’s just really straightforward.
E: Thanks! Unless I’m mistaken, you didn’t publish your book in progress, you published it when it was finished.
O: Yeah. I guess it was like a discipline thing for me, just so I would finish it first, and not put it out, and then get distracted with other things and never come back to it. So I decided to finish the whole book, and I think I do have a few chapters in my head now, that I’ve gotten some community feedback on the book. Kind of what people are wanting, or thought was missing, that I probably will go back and add a few chapters now. But essentially the book was published with all the material that I had originally conceived.
E: That relates to my next question - I guess you’ve answered it - but, is engaging with people directly important to you, and is there more we could do to help you do that? Or is that something you’re happy to manage on your own?
O: Right now, that’s been pretty easy enough to manage on my own, since I communicate with the community a lot through Twitter. I marketed the book mainly through Twitter. So people have been getting back to me on Twitter with feedback, and ideas for what they’d like to see, or what they’d like to see expanded on. It’s always hard to tell, you know, when you’ve written the framework, you have so many blind spots to what might be hard for other people to understand. It might make reasonable sense to you, but it could be clear as mud to everyone else. And so, it’s nice to get the feedback, because it’s very hard for me to ascertain what needs to be worked on.
E: OK, so you don’t find yourself wishing there was something more systematic, say, like what O’Reilly has, a kind of reporting system which is line-by-line, people submit errata and suggestions. Buy you’re happy, just using Twitter to interact with people that way, and I suppose keep your own list of things you’d like to change?
O: Hmm. It would be - you’re kind of putting ideas in my head now! - it would be cool to have almost a GitHub-style interface, where people who have bought the book can submit pull requests to fix stuff, that would be nice. But I’m sure that would be a lot of work!
E: Well, right now we have a very simple solution, which is Disqus comments that you can turn on. So there’s no, sort of, finishing them off when they’ve been completed, but it is a simple way to sort of focus things in one place.
O: That’s pretty cool. But mainly I’ve been managing it through Twitter, and people have been emailing me stuff to fix, and I just go ahead and fix it in the Markdown, hit publish, and you know, the update just goes out to everybody nice and smooth, so -
E: Do you watch Google Analytics for your book pages, or anything like that?
O: No, I don’t. I’m really bad about analytics! I don’t do a lot of analytics on Laravel’s downloads, or on the book, or anything, but that would be interesting to know - you know, who’s my primary buyers on the book.
E: Yeah, it’s a very interesting thing - given that your book is our third-biggest seller of all time, it might be a comment perhaps about, at least in the early months after publishing a book, what’s most important to focus on. Which is, getting out there to people, and maybe not watching obsessively one’s analytics every day.
O: Yeah, definitely. As a creator of a framework, I found it was easier for me to publish the book, because I had a fairly large Twitter following already. But yeah, it would be interesting to know, coming into Leanpub as a new author, that may not be well known, although you could have great things to say in your book, what the best ways to get the word out on the book would be. I’d be interested to learn more about that.
E: Can you tell me a little bit about what the book is about? I know it’s about 63 pages long, and I can see that it’s meant to be directed at people who are past the beginner stage.
O: Yeah, so, Dayle’s book was kind of the introductory, and I wanted to write a, “Here’s where you go next”, after you’ve read Dayle’s book, and got the basic grasp of the concepts of the framework, and now you’re ready to go deeper, and see what else the framework has to offer, in terms of building a more complex application. Something like Snappy, like we do at UserScape, where you have lots of classes, lots of stuff going on - how do you structure that, in PHP? PHP’s not really well-known for being, like, this well-structured and thought-out area of development. So we’re trying to change that, and give people some ideas for how to structure a complex PHP application, and particularly a Laravel application. But there’s lots of good info, I think, even if you’re not using the framework. We have five chapters, one on each letter in the SOLID acronym), which is a pattern of good software development -
E: Uncle Bob, right?
O: Yeah. From Uncle Bob. And then we have a few other chapters that are Laravel-specific - where to put your files, and how to organize everything.
E: I see that the book is very well-produced, including the cover. I’m wondering, did you go through a process whereby you had technical reviewers before you launched it, and did you hire say, editors, or something like that?
O: I didn’t hire anyone. I did have more prominent, or key members of the community review the book. I sent it to Jeffrey Way, who wrote the Laravel testing book, and of course I sent it to Dayle, who wrote the introductory book. And a few other people I communicate with a lot - I let them read it over, and give their feedback, and correct typos and stuff like that, or tell me I need to expand in certain areas. So that was kind of nice, to have a few members of the community that I could rely on there. I kept the book straight and to the point, which is why it’s fairly short, really. But I think the quality of the information is really good, even if the quantity is pretty short. And at the same time, I personally don’t like to read 200-, 300-page books, so I wanted to keep it fairly readable, where a person could buy the book, and actually finish the whole book, word for word, fairly easily.
E: Did you consider at any point going down the traditional publishing route?
O: Yeah, I did, actually. I got a few offers from other publishing companies before I settled on Leanpub. It just doesn’t make sense anymore, for me. With the framework being established, and with having a fairly large community, I decided it would be silly for me not to use Leanpub, because I would just be throwing away tons of money to write for a traditional publishing company. It just didn’t make any sense at all. It was harder to write, the platform was not as good - I couldn’t just write in Markdown, which was a killer for me. That was like a deal-breaker: I have to be able to write this in a simple Markdown syntax, because I can’t do some legacy Word crap, and just take forever - it would just suck.
Leanpub was an obvious choice, even though I had offers from, I think, Manning was my main offering, so…
E: Was timing an issue at all in that as well? With traditional publishing, well, I know that Manning has an early access program, but the traditional publishing process really does, can take quite a long time.
O: Yeah, I forgot about that. I was nervous about that, you know, like committing to the publisher, and being pressured to finish the book by a certain day. You know, I have a family, and I’m working on Laravel, and I have a day job - being able to write at my own pace was another big factor in that decision. There was almost no cons to using Leanpub, you know what I mean? It was more money, it was more time, it was no one telling me what to do, or what to write one, it was an awesome platform, you know, with Markdown, so it was like a no-brainer. There was really no question.
E: That’s really great to hear. That’s interesting, that one of the positive aspects, was the freedom to take more time than you needed to, because when we speak with authors, often they’re, especially when they’re people who are successful in one way or another, like you are, they’re often, they feel hamstrung by the amount of time that it takes with a traditional publisher. They feel that it’s a delay.
O: Yeah, for sure. I could just tell it was going to suck. I didn’t want to commit to a timetable because stuff comes up, I might have to work - something comes up in the framework, and I might have to spend like days on that, you know, don’t get to work on the book, and now I’m behind…. It was just going to be too much stress and too much pressure, I thought.
E: Have you been updating the book when you receive, say, typo suggestions and things like that?
O: Yeah, I have been. I’ve released a few updates. I’ve got a couple of emails of typos I need to go ahead and push out here soon. Yeah, I’ve made several updates to the book, to correct things, and fix any wording mistakes or example mistakes. It’s been nice, people have been nice to kind of report them, and stay on top of that.
E: Have you used the feature that allows you to email readers when you update the book? Or has it just been silent updates?
O: I think I’ve emailed on every update, and mentioned, Hey, this just fixes a few typos. It’s also a way to say Thanks for buying the book, and if there’s anything you see that’s missing, let me know. I use it was a way to communicate, at the same time, and not just to talk about the update. So I always remind people - if there’s anything you want to expand on, tell me, because that’s how I’m basing what any further chapters will be. So that’s been nice.
E: I see that you set your minimum price at $29.99 and your suggested price at $34.99. Was there any reasoning behind that?
O: Yeah. I priced it the same as Dayle’s book, and I also priced it a little higher, because the framework is free, obviously, it’s open source. And I felt like with the framework, I had made relatively little money off the framework, I mean, before I wrote for Leanpub, I may have made, I don’t know, let’s say one or two thousand dollars. I mean, not tens of thousands of dollars. And so, I kind of looked at the book as a half-donation, half-product so to speak. So when you look at the book, it is priced a little higher, but I think people have been really receptive to the idea of, Hey, I’m making money building apps for clients on this framework, and this is kind of a way to give back, and get something at the same time. It’s kind of a donation that has a reward to it. People have been really understanding of that, I think, even though it’s a short book, you know it is priced a little higher, but it is kind of a nice gesture toward the framework, and it will help us do some things down the road, actually, that we have planned, that might cost a little money.
E: Excellent. And have you used any of our features, like say the coupon feature, or anything like that? Or one-day discounts?
O: Yeah. I’ve used the coupon dozens of times. When I did release the book, I let people know there if there was a financial hardship, to email me and I’d give them a copy of the book, so, we’ve had various people, college students, things like that, that are kind of strapped for cash, but they want to learn about the framework. So we’ve given copies of the book to them. Also, at Laracon EU, we actually gave away 180 use coupons, so everyone at the conference got a free copy of the book.
E: Oh, fantastic!
O: Yeah, so coupons have come in very handy, I’ve used them quite a bit.
E: Do you have any specific feature requests? Something that you’d love to see, that occurred to you, when you were using Leanpub?
O: The only one that’s come up recently was bulk-creating a lot of different coupons. So like, with Laracon I needed to make 180 coupons, so what I did was make one coupon with 180 uses, but, if I could somehow create like 100 coupons at once, and just get a list of all the new ones, that would be great.
E: I see.
O: It wasn’t too hard for us to work around, I think it worked just fine. I don’t think anyone tweeted the coupon code to the outside world!
E: On that note, is that something you have a strong opinion about? Leanpub books are DRM-free, and people can actually, with two clicks, get a refund, within 45 days of buying. Did you think about those things at all, or are they - ?
O: No, I didn’t really. If people don’t like the book, I want them to have the refund, and if someone gets a refund because they just want their money back, I felt like that would be the minority. It’s not the way the community operates, in a sense. It’s kind of a very, I don’t know - like I said it’s a friendly community, I didn’t expect that to be a huge problem, people just buying the book and then say, I’m getting a refund and keeping the book. I didn’t think that’s the way it would be, so I didn’t worry about it.
E: And piracy isn’t a preoccupation of yours?
O: Not that I know of, we haven’t had any problems with it. No, I’m not really too concerned about it. I think people respect the framework enough, and what it’s done for them and their business. Really, thirty bucks is a drop in the bucket, I think, compared to what they’re getting out of it on a day-to-day basis, at least the people I’ve talked to.
E: Well, that’s great. A lot of the things you’re saying, are a lot of the reasons we built Leanpub, so it’s great to get the validation, knowing that it’s working.
O: Yeah, it’s a great platform, I totally love it, and if I ever write any more books, it will definitely be using Leanpub.
E: I guess that was going to be my second-last question: do you have any plans to write another book? You mentioned that you’ve got future chapters, probably to add to this one.
O: Yeah, I think I’ll stick with this book, for a while, and add some further chapters. I don’t want to - when I released this book I was kind of wary of, like, pimping the framework, so to speak - I don’t want people to think I’m trying to profiteer on the framework, so, I don’t have any ideas for releasing a second book, or really want to right now. I’m just going to continue to expand the book I have. And everyone gets updates for free, which is nice, so, yeah, I’ll stick with this one for a little while, and just expand on it.
E: Great. Well, I guess our time might be about up. Is there anything you’d like to add, or any comment you’d like to make?
O: No, it’s been great. It’s been great using Leanpub, I love it, and, you know, I think it’s really changed tech publishing in a way, in that, all these authors have access to a great publishing platform, and they get rewarded fairly for their work, and it’s just, I think it’s awesome - it’s awesome for the whole tech community, and I really commend you guys on the platform, it’s a really great platform.
E: Thank you very much, for your positive comments, and for being on the Lean Publishing podcast, and for being a Leanpub author!
O: Yeah, thanks for having me, it’s been great.
published Sep 17, 2013
Luc Beaudoin is the author of the Leanpub book Cognitive Productivity: The Art and Science of Using Knowledge to Become Profoundly Effective.
Luc is President of CogZest and Adjunct Professor of Education at Simon Fraser University. At Simon Fraser, Luc is leading the Cognitive Productivity Research Project, investigating knowledge worker cognitive productivity.
This interview was recorded on August 19, 2013.
Len Epp: I’m here with Luc Beaudoin, President of CogZest and Adjunct Professor of Education at Simon Fraser University, based in British Columbia, Canada. Luc is currently running the Cognitive Productivity Research Project at Simon Fraser, investigating psychological questions regarding knowledge worker cognitive productivity.
Luc has a Ph.D. in Cognitive Science from the University of Birmingham, where he conducted research on computer modelling of goal processing and motivation. Over the course of his career, Luc has held a number of positions in many very different fields. For example, he worked as a technical writer for Tundra Semiconductor, as a Senior Software Developer for Abatis Systems Corp., and he was Assistant Professor of Military Psychology and Leadership at the Royal Military College of Canada.
Luc is the author of numerous scientific publications on a wide range of problems in cognitive science. One of his most recent publications is his Leanpub book, Cognitive Productivity: The Art and Science of Using Knowledge to Become Profoundly Effective. In this interview, we’re going to talk about Luc’s research interests and his book, and about his experiences using Leanpub. We’ll also talk about ways we can improve Leanpub for him, and for other scientific and academic authors.
So, thank you Luc for being on the Lean Publishing Podcast.
Luc Beaudoin: It’s my pleasure.
E: Just to start out, can you tell me a little bit about how you first became interested in cognitive science?
B: Whoah, we’re going back! 1980s. I was a psychology student, and my career path was to become a clinical psychologist, and I was taking a neuroscience course, and that had me broadening my horizons a little bit, as university does to students. I was very interested in neuroscience, with this course I was taking, and I worked in a neuroscience lab. We were studying the neural basis of motivation and reward. So that basically opened me up to all kinds of possibilities. I took a philosophy course, an epistemology course - it was a requirement at the University of Ottawa - and there I discovered cognitive science, and that really intrigued me.
Cognitive science is basically the study of the human mind, and the reference model that we use for that, or the way of thinking that we use, is information processing. So it’s not that the human mind is a digital computer, but that it’s a device that could be understood as something that processes information. I thought that was very interesting. It’s also an approach to the human mind that recognizes that the human mind is too complicated to be studied just from one perspective. So there’s no one discipline that’s constitutive of cognitive science, that owns cognitive science. We often think of psychology as that which studies the human mind, but actually there’s many contributing disciplines to this cognitive science thing. So there’s artificial intelligence, there’s philosophy, linguistics, and other disciplines. Neuroscience contributes as well.
To make a long story short, I just fell in love with the idea. And there was a professor there, called Claude Lamontagne, who was just an amazing professor, and I was told “Luke, you have to take Professor Lamontagne’s course on perception”. I was going to take cognition, and I took perception in addition to that, so beyond the requirements I took that course, and I guess they knew that he and I would really hit it off. Which we did. He has a tendency to blow people’s minds, and that he did with mine. He basically convinced me that this was the way to understand the human mind, using information processing as a kind of metaphor.
E: Speaking of the connection between cognitive science and psychology, you’ve mentioned online that your Ph.D. thesis and your honour’s thesis “diverged from empirical psychology’s approach to the issues” you addressed in your theses, and you instead used a “designer approach”. Can you explain what you mean by a “designer approach”?
B: The idea is that if we’re to understand the human mind with this approach, we have to, in a sense, reverse engineer it. So we think, what are the requirements that this system implements? Typically we don’t look at the entire human mind. Some of us do look at the big architecutre of the human mind, but typically people focus on perception, or linguistic processing, and usually some very specific aspect thereof. But basically, we proceed as engineers would. First, understand what are the requirements of a system, and then we propose designs to satisfy those requirements.
And then we move on beyond that to developing computer programs that implement those designs, and then we actually press the Run button - the Compile button and the Run button - and we find out, well, this doesn’t exactly do what I thought it would. Actually it doesn’t compile. Or, more typically, you don’t even get to the point of being able to write the program, because what this does is it exposes gaps in your understanding.
But, you move along, and you do try to run these simulations, and then you can actually test your predictions in one way or another using your computer program. And then what you do is you study the relations between these different levels. So, by going through this whole process, you find out that you maybe didn’t understand the requirements properly, so you have to study that more in detail, your design wasn’t right. This is basically an engineering stance, which involves some reverse engineering to understanding the human mind.
Coincidentally, I had taken a psychological assessment my first year of university, in psychology. They said, “Did you every consider being an engineer?” And I thought, what are you talking about? I’m in psychology! I want to do therapy and understand the mind! Then, later I realized that this person, this psychologist, was spot-on. But I just didn’t know it was possible to study the human mind using engineering methods.
E: That’s fascinating. Is that still something that you carry on with, in your work for the Cognitive Productivity Research Project?
B: Yes. If you read my book, you’ll see that I do delve into that. As a matter of fact, I think it’s generally useful for people to have a working model of themselves, and each other, and to think of the components of ourselves and what they do. It helps us understand all kinds of things. Our emotions, each other’s emotions, how people respond, etc. We kind of have to do this anyway. Whatever reference model we use, we have to think about ourselves, and our behaviour, and what implements it. But cognitive science gives us some new concepts to think about ourselves, and to think about each other.
It’s a point I make in the book, that natural language has provided us with concepts and ways of thinking about psychology. That’s called “folk psychology” and it works very well, most of the time, but there’s all kinds of limiting cases. If you want to go beyond that, then you enter into the realm of cognitive science.
E: Actually, I wanted to talk about your book later on, so I’ll ask you some questions about that specifically. When you talk about some people’s misperceptions about the impact that technology can have on the mind. But before we do that, can I ask you a little bit about what CogZest does, and the story of how you founded it?
B: Sure. CogZest is an ambitious enterprise. The project actually started in 2002. It had a different name. I’ve switched domains many times, and you always bring some knowledge with you when you switch domains, but you also have to do a lot of learning. So as you mentioned, I started in psychology, then I was doing artificial intelligence, for my Ph.D. in Cognitive Science, and then I was teaching psychology at the military college, some courses that I hadn’t even taken myself, so I had to come up to speed quite quickly. That happens, that’s typical for a professor to do that. And then I worked as a technical writer, and all these different things.
So I’d gone through this intense, very intense series of learning cycles, culiminating at Abatis, where I was the first employee. We set up this company, and we had very ambitious objectives for it, and we were all just learning lots of stuff. I was the person who probably had to learn the most, in the early days, because of my bakground being different. They wanted somebody who was more of a startup expert, but I had done a startup before, and proven I could work on different parts of the business. So I was actually the first, and I had to learn about internet protocol, and routers, and all these things. I had to basically absorb lots of specifications, IETF documents, Internet Engineering Task Force documents. For a lot of people, including myself, that was new stuff at the time, the RFCs.
So I had to absorb this, as did other people, and it just occurred to me that the technology wasn’t really supporting us in the learning we had to do. We had these specs, and we’d print them out, so the printer was very very busy at Abatis, although the founders had ensured that we had the best technology. We had very fast computers, and we had the best monitors available, CRTs in those days. But still, we’d print out these documents, and we’d highlight them, and mark them up. So it occurred to me, as I was going through my learning process, at Abatis, that somehow, there’s something wrong with the browsers and the PDF readers. They’re not supporting our learning if we’re printing. So I thought to myself, you know, there’s got to be a better way. And coming from a cognitive science background, I had some ideas about what those ways might be.
After Abatis was acquired, and as things go, our parent company started to implode, so I started thinking about what do do next, and I had a lot of projects in mind. The one that appealed to me the most was, basically, bringing together cognitive science and technology to solve the problem of helping people learn, with cognitive science and with technology. My goal was to understand, first of all, what would be required. Again, the engineering approach, right? What would technology look like to better support our learning? As experts, now. I wasn’t looking at childhood or classroom learning. My interest since 2002, or 2001, has mainly been adult learning, thinking that then we can go back and help students with that, I think that’s true. So how can we help experts, at the top of their game, learn. So, I wrote some specs, and started a business - I was writing a business plan, all kinds of functional specs, that kind of stuff.
And then I met a professor called Phil Winne, who was studying self-educated learning from an educational psychology perspective. He had done a software project before, and he wanted to continue doing tha. And he was doing another one at the time, so he invited me to joing him, and we decided to join forces. I brought the Cogsci Plus technology, he had the educational psychology, and had done technology work as well, and we had business aspirations, and so we decided to work together, and we did. And we spent to the end of 2009 working together on various products, which did part of this. That’s the story.
E: That’s great, that’s exciting, startup life is fun.
E: Speaking of fun, I was looking over your list of publications, and I can see that you’ve coauthored the introduction to a forthcoming book called ‘From animals to robots and back: Reflections on hard problems in the study of cognition’. That’s a fascinating title - can you explain what the book is about?
B: Actually, I think the way I put it is that I’ve been invited to co-author, but I haven’t actually written that document yet. I’m co-authoring, is the idea here. What’s going on here is a volume of papers for Professor Aaron Sloman, who’s had his Festschrift, which is a celebratory conference, and a book being preparted for him, in 2011. He was my Ph.D. thesis supervisor, at the University of Birmingham, where I did my Ph.D. We decided that this warranted a book, and the book is a collection of papers from his various students, and people who worked with him, etc. What I’ve got, is a paper, the title of which I can give you - I think it’s on the website - it basically explores learning, learning with technology, so the project that I’m currently on.
E: You’ve also written about “super-somnolent mentation”.
B: You picked up on that!
E: Yeah, I had a lot of fun reading your list of publications. I did read a little bit of it, but could you explain what “super-somnolent mentation” is?
B: Funny you should ask! Again, you can think of this as a proof-of-concept for cognitive science. The problem that I address in that paper is reducing sleep onset latency. Many of us have limited contact time with our beds, whether we’ve got insomnia or not. Let’s say you give yourself seven or seven and a half hours in bed - well, let’s say you seven or seven and a half hours. In order to get enough sleep, you need to sleep all the while you’re in bed. Typically, it takes people 15 minutes, sometimes 45 minutes or more to fall asleep. So the problem I’ve been addressing, one of my background problems over 20 years, is, how could we reduce that sleep onset latency? And I think I’ve found a way.
E: Great news!
B: Yes, yes. Because it’s not just the initial sleep onset that’s the problem. By the time you hit 35 or something, you’re waking up in the middle of the night, and then you’ve got to go back to sleep. So then you’ve got to go through this cycle twice. So I thought, this is a really intersting problem, let’s solve it.
To make a long story short, I developed this theory, which basically said that if I could people into a state that’s very much like the mental state that they’re in as they fall asleep successfully, might be part way to helping them actually fall asleep. So that was one part of it, that was one angle on it. The other thing, I distinguish between different kinds of thinking that happen at sleep onset, or that happen at any time. I distinguish between “asomnolent thinking” - somnolence has to do with falling asleep - so “asomnolent” means that’s neutral, to sleep. Often we have “insomnolent” mentation, right? “Insomnolent” is the kind of stuff that keeps you up. So this is most of the pscyhology techniques aim to help you deal with. If you’re going to sleep thinking about problems, or problem-solving, planning, etc., that’s inconsistent with falling asleep.
E: So, overcoming a negative thing, that’s preventing you from getting what you want.
B: Yeah. To overcome that is counter-insomnolent. Most of psychology deals with counter-insomnolent stuff. So I thought, really, if we want to do something which is better, let’s combine the counter-insomnolent with something that actually takes you ever the edge.
Take meditation, for instance. That’s a technique that’s sometimes advocated for helping people fall asleep. Now, meditation, if done right, does not put you over the edge into sleep. It actually is a very focused state. You’re alert; you’re not supposed to fall asleep if you do it well. But it does deal with the insomnolent mentation. If you’re doing it well, you’re not bothered by these insomnolent thoughts.
So I propose that there are certain features of somnolent mentation that really take you over the edge. That involves having very disjointed thoughts and imagninings going on. And as a matter of fact, if you wake somebody up who’s falling asleep, you’ll find that they’re having these random memories and images and thoughts. Now, the trick is to get people into that state, and that’s one of the things I think I cracked with that paper. We’re actually working on a litte project that implements that - that helps people implement that. That’s going to be out very soon.
E: Speaking of disjointed thoughts, I’d like to return to your book. Sorry for having changed the subject abruptly, but you were bumping up against something, a great quote I found in your book that I wanted to ask you about. Near the beginning, you talk about “public epistemic overexuberance”. Maybe you don’t remember that line, but I really liked it! You’re talking about Steven Pinker and claims about how the internet can rewire our brains, by Nicholas Carr -
B: Hey, that’s pretty good, you found that in a footnote. Yes, I think that sometimes people get a little bit excited about neuroscience. I think neuroscience is great, it’s something to get excited about, but we sometimes get overexcited, and we lose sight of the way progress happens for the most part, in cognitive science. A lot of people are pushing brain-based education, and that kind of cognitive concept, and they’re not being called on it very often. So my book does a bit of calling, right? There are some things you can do at the brain level to take some barriers out of education, there’s no doubt about it. But I think it’s important to separate levels, just as we do with computers.
The computer metaphor is really powerful. It’s used at a high level, but it’s often not exploited as much as it should be. Because in computer science, you have very clear ideas of what levels you’re operating with. There’s the TCP/IP stack, a very clear conception of levels. In psychology sometimes, that gets blurred a little bit. You can do things to improve your brain - exercising is absolutely very important, it’s critical for brain health - and nutrition. But those are fairly indirect. So they give you some enablers, but the information processing level is different nonetheless. That’s what I’m getting at there.
E: It’s very timely. There was a great article in The New York Review of Books a couple of months ago, a review by Colin McGinn, of a book by a French neuroscientist, who was moving into the philosophical realm more than McGinn believed he should. It’s discussing exactly this - I just wanted to draw your attention to it, there’s a fascinating exchange between them in the Letters section of the latest issue where they’re talking precisely about what you’re talking about. It’s actually funny, to see the contrast between the neuroscientist, and the philosopher who’s calling him on a lot of these claims. That’s one of the reasons that the “epistemic overexuberance” which would perhaps apply not only to the public, but also to neuroscientists themselves often.
B: Without being critical of anybody in particular, I worked very hard on that footnote, because one has to be careful what one says. But the quote I have by Pinker there, I think he’s just got a great quote - I invite people to read the book, just to get Pinker’s quote.
E: On the braoder subject of the book, you talk about “meta-effectiveness”. Can you explain a little bit about that?
B: Yeah. Actually, that’s really what the book’s about. “Meta-effectiveness” is basically a matter of becoming more effective at ebecoming effective.
The main problem that I address in this book is, how do we become more effective at taking information on the input side - I refer to them as knowledge resources, be it a book, a podcast, a lecture, it could be any piece of information that has conceptual artifacts in it, so ideas that are packaged up by somebody to help us learn something. That’s the input side. How do we use that information to get to the other side? What is the other side? We’ve got to specify again, if we’re looking att requirements here, we’ve got to think of, well, what are we after? Often we just want to be able to remember something. Or we want to be able to develop some understanding of the thing. Or you want to develop a skill, if it’s a programming skill or whatever, and you want to be able to apply it when the time comes.
So in the book I look at these different outcomes, and I ask, how do we connect the input side to the output side? That’s a problem that we face when we pick up a knowledge resource. When we’re using knowledge resources, we often just want to solve an immediate problem with that knowledge resource. So it’s not as if we always want to use knowledge to become more effective in general, or disconnected from the resource. But sometimes we do, and we call that learning. Learning’s a word that’s used for all kinds of things, including this problem of developing understanding from information that we read. The book poses this meta-effectiveness problem: how do we become better at becoming effective?
E: Can you give us an example of one of the - I don’t want to give away the whole book, but maybe one of the techniques or tools for achieveing better meta-effectiveness?
B: Sure. The book deals with a lot of the problems we face just processing information.
I should maybe say something about the structure of the book, in order to put his into some context for people who are listening. In the first part, we characterize the problem, including this problem of meta-effectiveness, and the roadblocks that we face in solving this problem. In the second part of the book, I deal with the science that I think is most relevant to this problem. And in the third part, I deal with applications. There’s all kinds of concepts and tips and workflows and solutions for using information to become more effective. In the last two chapters, I really focus in on the problem of applying, and sometimes remembering, information.
I develop a concept there called “productive practice”. It’s an extension of some important concepts and findings in cognitive science, having to do with the importance of testing ourselves and learning. That’s called “test-enhanced learning”.
Then, there’s this concept of “deliberate practice”. So if you look at experts in perfomance practice, say a pianist or a golfer, so an athlete or musician or some artist who performs for the public - they do a lot of practicing, right? It’s essential for expert performance.
Now, knowledge workers also tend to practice in their own way, but it doesn’t tend to be as systematic. As students, a lot of us use practice, as a way of learning. they test themselves, they might read a chapter, and then write down questions that they need to able to answer without reference to the material. Then they test themselves on that. That’s quite a good way of learning content, and learning all kinds of things, like developing skills as well.
E: That’s a great connection, between something that we’re familiar with, such as practicing for sports or things we do with our bodies, but don’t necessarily think often about doing with matters of the mind.
B: That’s right. But again, students do this and they kind of have to. There are studies on this, and about 30% of students say they test tehemselves systematically as part of their way of learning.
But, student learning, one of the things that I draw attention to is that student learning is different from knowledge worker learning. For instance, students become very skilled at figuring out what’s going to tbe on the exam, what am I going to be tested on, etc. And then they adjust a lot of their mental work towards those aims. And that’s great. It’s good if as a student you can also think beyond that, and there’s ways of learning to get you down the long haul, but that’s a different story.
Many students will learn these tricks, and then graduate, but then practice kind of drops out of it. It’s not that practice totally drops out of it - we still practice. For instance, if you’re given a presentation, you may well rehearse it. If you’re learning a computer programming language you might do a whole lot of practicing to buiild up your skills.
But for the most part we don’t have a very systematic way of practicing, as knowledge workers. There’s many reasons for that. One is that there’s a lot of information to learn. Another is, so what do you practice? Another problem is, the tests of life are kind of flash exams, they’re implict, you don’t know exactly when they’re going to come, how they’re going to come.
Another problem is, it seems to me that the tool developers haven’t really made the connection that I’m trying to explain in this book, which is that to master information, and to master stuff, you really do need to practice, and systematic practice can be very beneficial. This is one of the things I realized in 2001 and 2002, is that we still don’t have these tools to help test ourselves on what we’re learning. I had previously developed tools like this, in 1991 and 1992, in Smalltalk, I had developed my first tools, because I fell upon this way of learning in the 1980s. I realized it was quite potent.
To make a long story short, if you can find a way to test yourself on key information, which I call “knowledge gems”, as you’re learning, then that could be very useful. One of the reasons why it’s useful, is that it changes your way of looking at a document. So all of a sudden you’re looking for knowledge gems. You’re saying, ok, what do I want to do with the information? Is there something in this document that is so useful that I could actually become a better person with this information, I could solve all kinds of problems on the fly with this information, or I could avoid problems. And I give some very potent examples from other researchers of things that, it really helps to know these things. So it makes you focus on these knowledge gems, and then it gives you a way of exploiting these knowledge gems. And that’s through practice, so to fully answer your question, it’s practicing with yourself, with challening, that’s the way to do it.
E: I was going to say, on the subject of making better tools, and people who are making them, you’ve got a tantalizing little story in your book about how once you wrote a white paper for Steve Jobs on how Apple could better support cognitive productivity in its products.
B: That’s right, yeah.
E: Can you tell us a little bit about that?
B: What happened was, because I had been working in this space for a long time - the space of using technology to learn - we all knew that approaching 2010, Apple was working on a tablet. That was no secret. Of course we were all speculating on, what would it be? So I thought, I’ve been thinking about this for a long time. I think I should document what I think it should be. So, I was interacting with the folks at SharpBrains, and I proposed, or they proposed, a paper, something on the iPad, and I said, I can do a piece on what the iPad should have, to support - I think at the time I pitched in terms of cognitive fitness, which actually ended up being titled “Brain Fitness”, which contradicts what I’m saying in terms of the levels, so I’ve felt uncomfortable about that title for a while. But really it was about cognitive productivity. So I wrote something for SharpBrains, and that’s publically available, saying, here’s some things that I’d love to see in the iPad.
And then when the iPad came out, it blew my mind. I was really impressed. But I thought, you know, there’s still some things missing. So I wrote about the iPad in light of what I had previously written - I wrote, it’s got a lot of things, but here are some things that are still missing. Basically, I felt that the iPad as a platform, just taking the iOS basically - it wasn’t even called iOS in those days - taking the iOS, the iPhone operating system and slapping it onto a bigger device, that just blew my mind, I thought, that’s so beautiful. And I thought, you can do tons with this, but it still hasn’t been done, so I then said, Why not? I can keep trying to do this myself, or I can go to the top here. So I offered some kind of collaboration with Steve Jobs, including a white paper, and he wrote me back and said, sure, send me a white paper on it.
B: I thought, that’s pretty exciting! I mean, people knew, if you were in the Apple world in those days, you knew occasionally Steve Jobs would answer people he didn’t know. It’s explained in his biography why he do that - he had a similar experience in the 70s, being on the asking end. So, I spent the most exhilirating week of my life writing a pretty hefty white paper for Steve Jobs, basically collecting a lot of my thoughts from the last ten years.
So I did that. But that’s not a public document. But I decided after he passed away, with different stories coming out, about how he approached innovation, I thought, this is a story that’s worth sharing. I’m not making a big deal about it, but it’s a nice little anecdote, and it speaks to a beautiful aspect of Steve Jobs, that he would accept to interact in this way with some person that he didn’t even know.
E: That’s a great thing to do. So, you mentioned some things are public, and some of your writings aren’t, so that’s a great segue into the next thing I wanted to ask you about, which is academic publishing. We’ve seen a couple of authors publish the Ph.D. and Master’s theses on Leanpub, but I’m quite sure yours is the first original book-length academic work. What are your thoughts on self-publishing and academic writing?
B: That’s a pretty general question.
E: I was trying to think of a way to put it more precisely. So I could say, Do you think that self-publishing is compatible with the traditional practice of academic publishing?
B: I think it really is a way forward for academics to do it. I think it’s a very significant way forward for academics to do their work. Let’s face it, the open access, it’s not exactly the same as open access - self-publishing and open access are not the same - but there’s some parellels, and they can intersect. Particularly with Leanpub, there’s no obligation, I don’t think - books can be free on Leanpub, right?
B: I take the question in particular in terms of what Leanpub offers, which is to develop important ideas, important documents, and to publish them as you’re developing them. Which, I think a lot of strong thinkers should be doing anyway. There are some, I think of Aaron Sloman, he’s my Ph.D. thesis supervisor, if you look at his website, he’s got these presentations and these documents that he iterates. It’s the same ideas, but they’re developing over time. You track his thinking as it’s developing over time. So he’s got these different documents, and you can trace their lineage.
That’s what a lot of people are doing, and that’s great. But it’s nice to be supported with a structure like Leanpub, in which it’s basically understood that, this is what’s going on here, that people are putting their ideas-in-progress on a particular site, and they can receive feedback and update their documents with feedback from their peers, or from anybody out there. It puts you in a mindset of thinking lean, which is, start small, iterate, and get feedback, and continuously improve your work.
If you look at what academics often do anyway, there’s a lot of rehashing that happens. Sometimes it’s the same idea being expressed in different terms, for a different publisher or whatever, but often there’s progress. So something like Leanpub I think really opens the door for academics to do this idea development in an open, honest, transparent fashion.
E: I’ve often encountered a stark contrast between the typical reactions of humanities professors and scientists, on the subject of in-progress self-publishing. Can you tell me about the reaction you’ve had from your colleagues when you tell them about your Leanpub book?
B: Actually, so far, I can’t say that there’s been a negative or a positive response, there’s hasn’t even been a big response so far. Yes, some people have said, if you want to be recognized, then you should put it through a decent publisher and get it published through there. But hey, I’m an adjunct prof, I’m not out there for recognition. And you know what? Maybe that’s important for the progress of science as well. But this book, while it has a lot of practical information, there’s a core scientific piece. This book really is proposing some new ideas as well. That aspect of things is not something that I’ve ever been too troubled about. I’m not a career publsher, so that’s not a big deal for me.
But I think, you know, a professor, or somebody who wants to be on the tenure track, probably, will want to be sensitive to those concerns. But one can also do parallel processing. To self-publish is not necessarily an impediment to doing another work using the traditional ways. And as you mention in your publications on the Leanpub site, there’s nothing in principle stopping from publishing their books with a regular publisher, after they’ve published it on Leanpub.
E: Yeah, that’s actually one of our hopes, that people will do that. And that publishers will start to see in-progress publishing that way. So that, instead of, sort of, passively waiting to be approached by agents, or people submitting manuscripts, or proposals for manuscripts, they’ll actually go hunting.
B: Yup. I think that the publishing industry is facing very significant challenges, and what’s going to happen is basically a process of economic natural selection. There’s going to be some publishers who are going to realize, doing this Leanpub way is really good, so let’s find a way to work with Leanpub authors. And it’s actually a great way to vet work, and to get some good content to publish. I think the publishing industry, and you’re a great example of this, is in transformation, and those publishers - there will be some more dinosaur-like publishers who will survive for a long time, but there’s a lot of pressure, and I think some of the publishers who are not adapting properly will have difficulty.
For instance, one of the big problems I have, is with platforms like Amazon and the iPad, even publishing for the iBookstore. I think of Kindle as an information jail. It’s so difficult to get information in and out of a Kindle book as a reader, and that’s a big problem. So in my book, I talk about how one could learn with PDF readers, and you could see that reading with a great PDF reader is a superior experience, compared to reading with, say, Kindle. To be fair, Amazon has improved Kindle, but it’s still a jail.
For instance, one of the techniques that I propose is to mark up a PDF file as you’re reading, to tag the key concepts that you’re after, or the important thing, important information. Versus, say, you’re knowledge gaps, things you don’t understand. Those are three kinds of things that from an educational perspective, there’s no doubt that it’s very important to do that. So you go through your document, and you can actually mark things up that way, and then when it comes time to review or take action on a document, you’ve marked it up, so you can go and you can quickly list, say, with a PDF reader called Skim, you can quickly list a document using the techniques I’ve described, say, to pull out your knowledge gaps.
Well, what’s the difference between an A student and a B student? After they’ve read a chapter of a physics book in grade ten, for example? Well, if you quiz these two grade ten students on that chapter, an A an a B student, odds are you’ll find that they’ll get a very similar grade. The A actually student won’t necessarily beat the B student on the first reading of a book, of a chapter. Where the A student shines, however, is if you ask him, What is it that you don’t know? What did you not understand? The A student has a pretty good idea of what he or she doesn’t understand. And that makes a lot of sense, because how do we improve ourselves, how do we come to understand things better? It’s not by focusing on, “Hey, man, this is the stuff I really got figured out, right?” It’s by focusing on what you don’t understand.
So if you look at the Kindle, where’s the tool for designating a piece of text as a knowledge gap? It doesn’t exist. Those tools aren’t currently available, but there’s ways of using, say, Skim, to do that kind of things. For knowledge workers, that’s very important. It’s also important for students.
E: That’s fascinating. It reminds me of Socrates - the wisest man, because he’s aware of his own limits.
B: That’s right.
E: On the subject of Leanpub, now that we’re nearing the end of the interview - can you tell me generally what your experience was like using Leanpub, and if there’s anything we can do to improve it overall?
B: I’ve really enjoyed using Leanpub. I had written my first draft, which was quite an extensive draft, using Scrivener, a tool that I really like. But then I realized, I want to do Leanpub - well I had realized, half way through this, that I really wanted to use Leanpub. I had not used Markdown extensively before, so I needed to convert the content from Scrivener to Markdown. Scrivener’s got a tool to generate Markdown, but I couldn’t work with that Markdown, that didn’t work for me. So, I ended up doing an HTML export, and your tool managed to import quite well, I was really surprised.
And at first I thought, well, this is not going to be that much work. But actually it ended up taking me several weeks. But, you know what? It was a big book. The Leanpub way of doing is you start with Leanpub. So, for an author’s who’s starting in Leanpub, he doesn’t face that, in Markdown, he doesn’t face that problem. This is not a small book, you know, so I had a lot of content. But I did reserve some - there’s a chapter that I started, that I’m working on from scratch, on the Leanpub platform, so that’s the deliberate practice chapter, that’s not complete. That’s chapter seven, and the conclusion, because I want to go through that experience of really adding new content with Leanpub. So, what can I say? Just generally I can point to the fact that that was not as easy as I had hoped it would be.
Now, in terms of other things that I would like to see in Leanpub… Right now, for some reason I’m drawing a blank… I had proposed certain things, I’d like there to be an even better PDF reader, and I thought, this would fit so naturally with Leanpub, but that’s not really targeted at authors, that’s more targeted at readers, and I realized that Leanpub is really focused right now on supporting, getting authors to get their content out.
E: That’s true, although we’re now expanding our focus a little bit to be more reader-centric. It’s sort of the next stage in the evolution of Leanpub. In particular, we see the relationship - the idea of in-progress publishing like this, is relatively new, and it’s a way of potentially establishing a new kind of relationship between authors and readers. Authors and readers have always communicated with each other, but not as it were in real time, on the subject of an evolving book. So maybe that might - is there anything along those lines maybe that you’d like to suggest?
B: Well, here’s one thing, and we’ll come back to that particular aspect, before I forget the other thing.
As an author, I keep track of my release notes, but I haven’t actually published them yet. I published them once, but I didn’t like the structure, so I’m going to come back and let people know what’s changed between revisions. Now, O’Reilly, as you know, has this feature - I don’t know how they do it, but it must be self-driven, it is O’Reilly, after all - where the author, in a systematic fashion, is able to indicate what’s changed in each revision. So, right now, as an author, when I publish a new revision, I have a choice of emailing all my readers at once, which I don’t like to do, you wouldn’t want me to do that either. I’ve already got I think over 60, maybe over 70 releases,of my bookk.1 Because I love lean, I love this idea, I embrace lean, I think it’s great. So although I did a “Big Bang” integration with my first integration into Leanpub, but since then I’ve just been doing these minor modifications as I go along. Why not? If I find a typo, why should I wait? I just press the publish button. Or, more significant changes, I’ve done all kinds of changes already. But I’d like to systematically be able to indicate the problems to my readers. I’d like to have a form, or, even better, in Markdown, or a spreadsheet or something, a tab-delimited file or something where I can say, OK, chapter, page number, original text, new text, description of change, attribution, you know, to thank whoever found the error. Something like that would be cool.
E: Attribution, that’s a very interesting idea. It would encourage people to respond, and give them some positive feedback, a positive sense of participation in the project.
B: Yeah, and then readers could go, and they’d have access to this release note on the website, they can see what’s changed. Then they can get a sense of the book, this is an evolving, you can look at the velocity of change, and it’s cool. I’m sure you’ve got all these data in your Git, so there’s endless possibilities, even scientifically, for you, with all your data. So that’s one thing.
The other thing I would say, maybe getting back to your reader concern, is helping my readers, who would adopt the ways of working with PDF and other formats, that I propose - say, a user marks up a document, using Skim, which is a PDF reader - he marks up a book, say, finds several key ideas, marks them up, and then I come out with a new revision, and then what? Well, his annotations are now lost. So here, Kindle wins. I don’t know how Kindle handles updates, but presumably they try to anchor and re-adjust the annotations. So a framework like Diigo for instance, does that all the time - Diigo’s a web annotation tool, so you can go to a web page and highlight or make annotation or add notes - and if that web page changes, then Diigo’s got some logic to take a best guess at where the annotation should be anchored. I’ve developed tools with teams of software developers previously to do this kind of thing with web content. So, right now, the PDF readers such as Skim, they’re not set up for this, but if you wanted to get into that, then I think it would be very important for this tool to be available.
E: So what you would like to see, perhaps, from us, is something of a metadata kind, that an external app, designed for annotating, could then use in order to do things like appropriately anchor a note, in a section that’s actually been deleted, or something like that.
B: That’s one way you could do it. You could perhaps provide the metadata, that would be one approach. The other approach would be to collaborate in some other way with a small business that’s developing an enhanced PDF reader, that would handle this aspect, or maybe other problems. What’s the big picture? It’s to allow a PDF reader to do continue to delve an evolving document. “Delve” to me means “actively read”, in this context.
E: And is there anything we could do to help you discover more readers, or help readers discover you, that occurred to you as you were launching your book, and starting to tell people about it?
B: I don’t necessarily have solutions, but I’m acutely aware of the problem. That is an issue, right! But I’m sure it’s one that’s going to be solved. I thought, well, maybe connect is one thing that could be done, maybe on the service side, is connecting service providers, editors or book marketers or whatever, who you’ve vetted in some way, that would help in the book marketing process.
E: Ok, like Leanpub-approved cover designers, and things like that?
B: Right, yeah, and even on your site - I don’t know how integrated you want to do it, but something in that direction.
E: Ok, that’s a very practical suggestion.
B: I’m just very happy, I’m really pleased with Leanpub. I’ve recommended it to other authors, and I’ve had a great experience with it.
For instance, the bibliography thing. We talked about Leanpub for academics, and bibliographies weren’t being formatted properly because Markdown doesn’t handle that. Within a couple of emails, the feature implemented, and then all of a sudden, bang, my glossaries and my bibliographies are up to a standard that they’d be if I was publishing this with anybody else.
E: It was hanging indents I think that Scott implemented for you.
B: Hanging indents, yes.
E: Thanks for all the positive comments and suggestions. I have one last question. Are you planning on writing another book?
B: Oh yes, we’re quite ambitious actually. There’s a follow-up to this one, which would be focusing more on the practical stuff. Taking ideas from chapter three and re-presenting them, and extending them, minus the science.
E: Ok, so the techniques and tools.
B: Yeah, so there’s that. We’ve got sleep products coming out, and there’s a need for a book on sleep, specifically on sleep onset acceleration. So things like that. I think I will write more, with co-authors, perhaps. I’m looking forward to it.
E: I’m looking forward to reading the full article on how to fall asleep! That’s very good. Ok, well, thank you very much Luc for being on the Lean Publishing Podcast, and for being a Leanpub author!
B: Thank you very much, and we’ll be in touch.
Correction: I have only made 39 revisions so far, not 60+. I think I was remembering the number of readers! (69!) - LB↩
published Sep 04, 2013
Paul Bradshaw is the author of several Leanpub books, including Scraping for Journalists, 8000 Holes: How the 2012 Olympic Torch Relay Lost its Way, and Data Journalism Heist.
Paul runs the MA in Online Journalism at Birmingham City University, where he is a Reader in Online Journalism at the School of Media. He is also currently a Visiting Professor at the School of Journalism at City University, London.
This interview was recorded on August 15, 2013.
Len Epp: I’m here with Paul Bradshaw, who runs the MA in Online Journalism at Birmingham City University, where he is a Reader in Online Journalism at the School of Media. He is also currently a Visiting Professor at the School of Journalism at City University, London. Paul has written and contributed to numerous books, in addition to his work for journalism.co.uk, the Guardian and the Telegraph’s data blogs, and other media organizations. In addition to running the Online Journalism Blog, he is the founder of HelpMeInvestigate.com, a platform for crowdsourcing investigative journalism. Paul also works as a freelance trainer and speaker.
Paul coauthored the Online Journalism Handbook, and the third edition of Magazine Editing: In Print and Online, both of which came out in 2011. He is also the author and coauthor of a number of Leanpub books, including Scraping for Journalists and 8000 Holes: How the 2012 Olympic Torch Relay Lost its Way, and Data Journalism Heist.
In this interview, we’re going to talk about Paul’s work and interests, as well as the subject and development of his Leanpub books. At the end of the interview, we’ll also talk about his experiences using Leanpub, and ways we can improve Leanpub for him, and for other authors.
So, thank you Paul for being on the Lean Publishing Podcast.
Paul Bradshaw: Thank you for having me.
E: My first question is, how did you become interested in journalism?
B: Well, I guess I grew up reading newspapers and being very much aware of the news as a kid, and I was seen as quite unusual at college, as being someone who read a newspaper. So, I obviously stood out in that sense. That probably says more about the college I went to, than me! I was really more interested in writing, and actually art, when I was a kid, and I kind of drifted into journalism, so to speak, because I could write and that was a job that I ended up doing.
I actually wanted to work in radio, I didn’t have that high an opinion of journalism when I was first working in it. But what I enjoyed when I became a journalist was that ability to hold power to account, and that you could represent your readers and get results in a way that they couldn’t always get themselves, individually. So that’s really what gave me a bug, if you like.
E: What was your first job in journalism?
B: I worked for a publisher when I was about 17, just as an admin role, I worked for another magazine when I was about 19 in a similar admin role, but my first full journalism role was a magazine, reporting on the internet, and I was very lucky to get a job as an editor, and I grew to become group editor, and other editorial operations.
E: Why did you eventually make the switch to academia?
B: I wanted to leave the job I was in. I’d completed one major project, of building a series of websites, and I wanted to move one, and a job came up at the university, what’s now Birmingham City University, and I originally applied for that with the intention of it being a bridge to a freelance career. But I actually really enjoyed teaching and learning new skills. I never planned to go into academia, it was something that I did initially as a bridge to something else. I still enjoy now learning new skills and trying to communicate those in the books.
E: You run the MA in Online Journalism, as I mentioned in the intro, at Birmingham City University, and I watched a video in which you say it’s about “defining online journalism” and “shaping the medium for the twenty-first century”. Can you talk a bit about what the course offers, and how it’s different from what journalists were taught, or needed to learn, maybe even just a few years ago?
B: Well, I think a lot of journalism education has been set up to meet quite a uniform news industry. And that’s all changed now. The news industry is trying to reinvent itself, in reaction to different ways of consuming information, and different ways of advertising as well. So, what I wanted to do with the MA was, I guess the realization I had was that sending out students with identical skills was not going to help the news industry or the students themselves. So, the people who are going into the news industry now are the people that are finding out how it can be different. They’re the people who are innovating in forms like audio slideshows and live-blogging, the people who are doing investigations with data journalism, the people who are engaging with online communities.
So I wanted to create an MA which was worthy of a Masters-level certification and allowed students to define the medium and to push the boundaries of the medium that they were working in. The way that it does that is it gives students an introduction to a broad range of key skills in online journalism, but gives them a bit of a nudge to experiment and to produce high-quality, in-depth work.
So there are parts of the course where they have to experiment. There’s an element which is called an “Experimental Portfolio”. It’s a space where they can try something that might be intimidating to them, but they’re not going to necessarily fail if they don’t pull it off, as long as they learned something in the process.
And alongside that there are more traditional portfolios of work. There’s a lot of working with clients, so they take industry problems and research those and try new ways of addressing those, and that might be anything from, how do you make online videos successful, to how do you train citizen journalists. And they all come out with very different skills and unique representations, a lot of them build reputations on the course, and so far every single one of them has found a job, which is incredibly unusual.
E: That’s fantastic.
B: Most of them at strategic levels as well, which is also encouraging. So they are genuinely going into positions where they are shaping where the news industry goes next.
E: That’s fantastic. Can you explain a little about what “data journalism” is?
B: Data journalism is quite a broad range of skills, really. In a nutshell, it’s using data, structured information, at some point in the process of getting or telling a story.
That’s quite a vague definition, but in practice that means someone could be collecting data in a way that might be hard for other people. It might be that they might tell a story about that data, or public data, again in a way that other people aren’t doing. Or it might be that they use new storytelling techniques like data visualization, interactives, databases, things like that, to communicate the story. Like I said, there are quite a wide range of skills involved potentially, and quite often there are collaborative projects that allow different people different skills to do that.
E: Is that also what you mean by “computer assisted reporting”?
B: Really, it’s grown out of computer-assisted reporting, for me. So, computer-assisted reporting came to prominence in the US in the late 60s, through into the 80s and 90s, and for me that’s using spreadsheets and databases to find stories. The difference with data journalism is that you’re now starting to do that in a networked environment, so you might be accessing data on other computers, you might be allowing users to access that data and do things with it, and make maps and so on. So it does include what traditionally has been called computer-assisted reporting, but I think it also takes in a number of other techniques. You’ve got design skills, you’ve got web development skills, that traditionally, are new things really, I guess.
E: In your Online Journalism Blog, you mention that you support experiments with “wiki journalism”. Can you explain what that is and why it interests you?
B: Well, there was a period, probably a couple years after Wikipedia came to prominence, when journalists and news organizations were experimenting with wikis as a way to, again, tell stories and engage with audiences. That was, I did some research at the time into what worked and what didn’t, and different ways of doing that.
It has been a really interesting way for news organizations and journalists to collaborate with users. You’ve got all sorts of knowledge out there in what used to be an audience, a traditional audience, which now you can access and publish as a news organization, and wikis were one of the earliest attempts to do that in some sort of structured way. And you had some interesting examples where newspapers would say, “We’re going to do a wiki on our local music scene, and can you help us add entries for all the bands that you know of, and the venues”, and all the rest of that. It’s something that you wouldn’t necessarily do as a journalist, but is a tremendous community effort of sharing information and expertise, for something which everyone benefits from and enjoys.
I think you’re seeing less of that now. I think the novelty of wikis has worn off, and people are interested in other things now. But potentially still, I think increasingly you now get specialist, either wikis, or people use Wikipedia. So for example if you’re a Lego fan, then there’s Brickipedia for you, and if you’re a Star Wars fan, then there’s Wookieepedia. There are all sorts of specialist wikis now, and I guess if you’re interested in one of those areas, you will contribute to one of those wikis.
E: Can you talk about helpmeinvestigate.com, how it got started, and some of its successes?
B: This came out of some research that I did for a book chapter. I was asked to do a chapter for the second edition of a book called Investigative Journalism, which is edited by Hugo de Burgh, and they asked me to do a chapter on investigative journalism online. And in the process of looking at that, and writing this book chapter, I looked to a few examples of crowdsourcing, and it really struck me as - by the way crowdsourcing is the idea that you gather information from a number of different people, so you allow users somehow to contribute to an investigation, or a story - and when I looked at crowdsourcing and investigative journalism, I found some really interesting characteristics about the engagement, for example.
There was a particular example of an investigation by the Florida news press, which had tremendous levels of traffic, the traffic on that story was higher than pretty much any other story on the site, that they’d ever done, apart from hurricane season stories. So, that’s relatively unusual for investigative journalism. Yes, you get big scoops, and yes they make a lot of noise, but in terms of how many people actually read that, you’re always struggling with investigative journalism to get people to care about something that’s important. So, the engagement around that was really interesting.
And the other factor that struck me was you could use crowdsourcing to do investigations that traditional news organizations would not normally do, in other words, unfashionable subjects, subjects that didn’t have a mass audience, or that didn’t warrant a particular amount of effort. And so it opened up new possibilities editorially. That kind of led me to come up with the idea for Help Me Investigate, which was really a way of systematically testing, how could you crowdsource investigative journalism, and was there a way of doing it more successfully? You know, what would be the techniques for doing that?
And after a year or two of developing the idea, I got some funding from Channel 4, a television company in the UK, and we launched that in 2009. It’s been running for four years now, in fact it just celebrated, although without any noise at all, its fourth anniversary, and we’ve just been investigating different questions, anything that people raise or approach us with, and it’s just a network of people. It’s completely non-profit, it’s completely voluntary, there is no business model because there are no costs. But we work with news organizations including the Guardian, the BBC, local newspapers, newspapers in Germany. We’ve done all sorts of different things with different organizations, and it’s really just a way of trying to use network technologies, the internet, to put people in touch with other people who might want to help each other investigate something.
B: It doesn’t surprise me at all. In fact, funnily enough, I was predicting, for the last two years I’ve been saying that Facebook, Amazon, or Google will start buying news organizations at some point. What surprises me is that it isn’t Amazon, it’s Bezos. But having said that he’s not the first, I can’t remember who it was, but one of the founders of Facebook bought one of the news magazines in the US, I can’t remember which one. There are other examples of either people who’ve made their money from the web, or organizations, web companies, buying traditional media properties. I think that’s going to continue.
It does open up some interesting possibilities in terms of, what he’s going to actually do, and that’s probably more interesting than the sale itself. But, any print organization I think has got a very difficult challenge in trying to straddle new opportunities, and where things are growing, while also not losing the massive revenue that still comes from print. And also, ultimately, they’ve still got people in jobs that they can’t just chuck out the window, so you’ve got a whole workforce which is something you’ve got to work with.
E: So you think this is a potentially good thing?
B: I’m always hesitant to say something is good or bad, because it’s always going to be mixed. So I guess that would be my prediction, if anything. I think it is a thing, which will have both good and bad consequences, and we will always make mistakes, particularly with change, but we learn things along the way.
E: Switching to the subject of your books, the first book you published on Leanpub was Scraping for Journalists. Can you explain a little bit about the book, like for example what “scraping” is, and why it’s important for journalists?
B: Scraping is the process of getting information from a series of web pages or documents online, or even just one web page. And how you get that information from that web page or those web pages normally involves creating some sort of script, which is a set of instructions. So that’s scraping: scraping is essentially writing a set of instructions to grab information from the web.
Now, there are a number of ways you can do that. You can get tools where you click buttons that write those instructions, and you tell it what information you want, and where it is, and you kind of press play, and off it goes and gets that. But if you want to get something that’s more complicated, or for example it’s in PDFs, or it’s in spreadsheets, then you might have to actually write something more advanced, and write some programming.
The book really takes you across those tools and those skills. It starts from a very simple script, which is just using Google Spreadsheets to grab information from a web page. It takes you from that to the kind of click-and-play tools that will scrape pages for you, and then right through to learning some programming to write scrapers.
Now, the reason that that’s important, is that obviously there are millions of pages of information out there that are potentially of interest to journalists, and a lot of those pages include government data, information about health, education, all the sorts of issues that we’re supposed to be reporting on as journalists. It allows us to check facts, it allows us to identify problems, it allows us to identify trends, it’s not just hard news, it can be softer subjects like sport and fashion, and quite often scraping is a really useful technique for getting hold of that data, when you either couldn’t get a hold of it in any other way, or the alternative methods take a lot of time, like Freedom of Information requests. So that’s really what scraping is about and why it’s important.
The other part of the book is, I’d kind of been trying to teach this to journalists for a while, and I’d found that there was a disconnect between how journalists saw scraping and how programmers saw scraping, and basically journalists quite often come from a humanities background where they learn in a particular way. Programmers, I’ve noticed, operate differently. They’re more like scientists. They work through trial and error. They borrow from each other. They don’t expect to learn everything, and then that body of knowledge is all they need to know.
E: It’s interesting, you have a great line in your book where you say, “If you were used to getting things right first time in school and college, get unused to it”.
B: Yeah, absolutely. That was the hurdle, I think that I saw in doing this in training, and also learning it myself. I’d been trying to learn scraping, and I’d read books about programming languages, and I’d be kind of like, “Well, this is great, I know how to write a script that adds two numbers together, but how does that help me get data from that web page?” And I could see that journalists, that they want results, and they’re not going to read a whole book on a particular programming language, and then feel satisfied that they can write a scraper.
So it’s as much about that as anything, about a way of learning, and about saying, forget the idea of getting everything right, you’re going to get things wrong, and actually the fun of scraping - and I really enjoy scraping, because it is like solving puzzles, and the fun is making mistakes and solving them, and coming up against problems - it’s Sudoku, really.
E: I have what might be a very specific question about that, actually. I was wondering, for example, if you wrote a story based on some data that you’d scraped, and then, let’s say, one of the subjects of the story later changed the data on their website, how would you establish some kind of audit trail, proving where you got the data from?
B: That’s a really good question, actually. I imagine the audit trail would be the scraper and essentially the metadata that that scraper recorded at the time of being run. So you would be able to prove that, this scraper ran at this time, and it grabbed this information from this URL. Really, if this was a court case and they had to seize computers, or whatever, those computers would be able to back up that information, and it would be very - say for example the other party says that you falsified that, again, they could check, well is there a way that this could be falsified, and all these logs have not been manipulated. So that would be, that’s getting into too much depth really. All you’d have to do, practically speaking, is going on Google and search for a cache, or go on the Wayback Machine. Or their computers would be seized, and you’d be able to look at the previous versions of the pages that were on, and when changes were made, and so on. So there’s always going to be an audit trail on both sides.
E: In the introduction to the book, you mention that it’s being published in progress, and that readers can influence what gets written and how. Can you give me an example of how reader feedback has influenced what you’ve included in the book?
B: This week I’ve had a tweet from someone who said that there was a particular page that I’d used as an example in the book which was no longer online. So I went back in and I’ve changed that passage in the book, now, so that doesn’t include a mention of that. Someone else suggested all kinds of alternative ways of creating a particular process in the book, so I’ve added quite a lengthy section at the end of that chapter which says, this is what this person says, here’s another way of doing it, and it’s much better because of X, Y and Z. People have picked up typos, people have picked up things that are changed, people have made suggestions of different ways of doing things.
Another thing that’s happened is new tools, people have suggested alternative tools. I had a great case where someone in, I think it was Portugal or Spain, where they said that Google Docs or Google Drive, in their language, doesn’t use commas, it uses semi-colons. So you get these small idiosyncrasies that I added into the book. There’s a little alert box, saying, “If you’re using the Spanish or Portuguese (and there’s another language as well, it may have been German), then use a semi-colon instead”. Just small things like that, which you’d never be able to do in print.
E: That’s great. And the kind of thing, especially if you’re learning programming for the first time, the kind of thing that could really frustrate someone. Because you see you’re copying what you’re being shown, and you know you’re doing it right, but even the difference between a comma and something else can break everything that you’re doing.
B: Exactly, yeah, and even, you can try and anticipate people making mistakes, or misunderstanding, or having problems, but having people raw testing the book and saying, “This doesn’t work” or “I made this mistake at first”, so then you can add just a little passage, or then you might want to rewrite a sentence so that it’s clearer. That makes a big difference.
E: Your second Leanpub book, which you coauthored with Carol Miers, was 8000 Holes: How the 2012 Olympic Torch Relay Lost its Way. Can you tell me what led you to write this book, and what it’s about?
B: It’s a great story, at least to me, I think it’s fascinating. This came out of scraping. To cut a long story short, I scraped details on the Olympic torch bearers, and there were 8,000 of them - this is for the 2012 Olympics in the UK - and the official website published details on all of the torchbearers, so I scraped that information and started to investigate all sorts of things about them. So we did dozens of stories about the torchbearers, everything from who was the oldest torchbearer, through to these stories that appeared on the front pages of the national newspapers, about corporate torchbearers.
The main story that we told as a result of this was that a number of corporate sponsors of the Olympics had nominated their own executives to carry the Olympic torch. What was important about that, was that when the Olympic torch relay was announced, in London, they made a promise that it would be about people who were inspirational, who had sporting achievements, who were young - 50% of the torchbearers were supposed to be young - 95%, I think it was 95% of the torch relay places were going to be made available to the public. It was very much about celebrating people, everyday people, inspirational people, and so on. It wasn’t about executives. And in fact the Olympics minister said that they had issued guidelines to these sponsors saying, do not nominate executives.
Now it turned out that dozens and dozens if not hundreds of executives carried the torch. And it was clearly being used to build corporate relationships. Corporate partners got them, and so on. And that was a story that I found early on in the process of looking at the details of the torchbearers. But as the torch relay progressed, I wanted to tell a deeper story about how that had happened. Because, it’s one thing to say, these corporate executives are carrying torches, but it’s supposed to be for normal people, but it’s another thing to actually tell the story of, why did that happen, and why is it important?
E: You tell a very moving story in the book about an athlete with brittle bone syndrome, who ended up not being able to carry the torch, and then discovered that there were all sorts of people who hadn’t done, and he had to go through a very long process of being nominated, and rounds of assessment, and was disappointed to find out that he wasn’t included in the relay, even though a bunch of people who hadn’t done anything nearly as sporting or inspirational were included in the relay.
B: Yeah, and that was what it came down to. As the torch relay came towards its final weeks, I wanted to write the full story of why this mattered and what had happened. So we started the book with the story of Jack Binstead, who was I think 17, so he fitted the criteria of youth, he had broken 60-odd bones since he’d been born, and he was probably going to compete at the next Paralympics. He was nominated by the maximum number of people, but he didn’t carry the torch, and as you say, on the day that he probably would have carried the torch, there was this long list of marketing executives and chief executives and political donors, all of whom were carrying the torch instead. And, I mean it’s sad, but I recently found out that he’s retired from the sport, he’s not going to be competing at the Paralympics, he’s decided to quit wheelchair racing, and you do wonder, had he been successfully nominated to carry the torch, would he have retired?
So, that was why this story mattered, and I’m glad you think it’s moving, because it certainly moved me, and moved me to set up the book on the site so that people could get it for free, but any purchases went straight to the Brittle Bone Society, and Leanpub was kind enough to volunteer to donate the Leanpub share to the Brittle Bone Society, so 100% pretty much goes to them. And we also closed the book with the story of a person who did carry the torch, but also found the experience sullied by the fact that all sorts of people who carried the torch when he was, got to carry the torch because they sold the most bottles of Coca-Cola that month.
E: It’s that kind of thing that moved me about it, the contrast between a person who had strived to do things of Olympic kinds in order to carry the torch, and then the, I don’t know, the kind of ugly vanity of someone who’s merely doing it so they can brag about it at a dinner party or something like that.
B: It’s a shame really, and even after all of that, there were probably a good thousand or so torchbearers who were never named on the site. I know at least one of those is an executive at a sports retailer that was never named, and never identified, and his two daughters managed to carry it as well.
B: It’s a family affair. But yeah, it was a really interesting exercise to try to tell the full story and detail all the various twists and turns of that.
E: I can see you’re working on a new book, Data Journalism Heist. Can you talk a little bit about that?
B: This is me trying to keep it simple, after doing ‘Scraping for Journalists’, which was very, it’s quite a long book, it takes you from a very simple starting point to somewhere that’s quite advanced. And I wanted to do something which was very introductory, very basic, very short, was for people who were nowhere near scraping really, and were just interested in data journalism generally. So it’s going to be a very short book which teaches a couple of simple techniques in spreadsheets to find basic stories in a simple dataset.
It’s probably more like a very long article than an ebook, and the idea varies, but hopefully a lot people will use that, will get interested in data journalism, and then they might want to start to explore some of the more advanced techniques like scraping. I’m hoping to do some other books around other data journalism techniques as well.
E: Switching gears a little, I’d like to ask you some questions about your experience with Leanpub and the Lean Publishing process. Can you tell me how you found out about Leanpub, and why you chose us for your publishing platform?
B: A colleague of mine at Birmingham City University, a guy called Andrew Dubber, he used Leanpub to publish, well he still uses Leanpub to publish all sorts of books on the music industry, and I’d been looking at ebook publishing for a while, and he’d mentioned it in conversation. I liked the idea that you could continually publish, and that was the main selling point for me. I’d obviously looked at other ebook publishing platforms and so on, but the way that this seemed to be designed to allow you to start publishing and then keep adding to something, that was the key thing for me, particularly for something where technology is changing. In fact, I myself was still making that journey and still learning things. So that’s how I got into it, really.
E: And did you know Markdown before you started using Leanpub?
E: What was the experience like learning it? Was that a barrier to you, or was that just something you picked up easily?
B: It was a barrier. Not a huge one. I think I’d stumbled across it a little bit, and I don’t think, I don’t know if I’d ever used it. It was something that made me think, well that might stop a few people. Occasionally it will still trip you up. I think it’s, once you’ve kind of got the basics of it, it’s largely just things like images and having to remember, I think you occasionally have to think, “How do I do an image again?” And you have to look at another chapter that’s got an image and just copy the syntax and paste it into your new chapter.
And I only recently discovered the Asides, the different things that you could use to create information boxes, or question boxes, or discussion boxes. So I went back all the way though the book finding things that were, you know, “that’s really an information box, I’m going to change that now”. So yes, there were all those hidden gems as well. That’s probably more of a selling point, actually, the special codes for Asides.
E: And Markdown of course is a great way of making it so that you can write one single source text, but then output in multiple ebook formats, and even HTML of course, which is what Markdown was designed for.
B: Funnily enough, I use it now for other things as well. I’ve just finished writing a book chapter for a print book, and I wrote it in Markdown, copied it into a Markdown converter, and then copied it into, if you like, a word processing document, because it means I can type it on the train, just in a text editor, and not have to think about that.
E: That’s great for us to hear! I see you have a Facebook page for ‘Scraping for Journalists’. Is engaging directly with people who have already bought your books important to you, and is there more we could do at Leanpub to help you engage with your readers online?
B: It’s definitely important to me to engage. I guess the Facebook page doesn’t do a lot of that. It is a channel for people to point out things, or ask questions, but because I’m a lot more active on Twitter, I tend to get people contacting me, or occasionally through email, so the Facebook page is really there as a point of contact, more than anything else, and I’ll just publish anything that I find about scraping will get published on there, but nothing much else happens there I guess. Although I did use it to announce the new chapters, and I guess that’s probably where it could be better integrated in Leanpub, in that you could say, “when I publish a new chapter, don’t just email readers, but update Facebook, or update the Twitter account if there’s one for that”.
E: That’s very interesting.
B: So, for example, I’ve got a Twitter account for the ‘Online Journalism Handbook’, but I didn’t think it was necessary to create one for ‘Scraping for Journalists’. Having those different ways of being updated, and then also, as part of that, thinking about it, when someone buys the book, they obviously make a choice whether to get updates. Now they could choose to either get that via email, or they could follow the Facebook page, or they could even follow the Twitter account, from that single point. That might be a different way of approaching it.
E: That’s very interesting. Is there anything generally that you think we could improve, or any other features that you’d really like to see?
B: I guess, I had to go off to find a Markdown editor, converter-type-thing, so having something on the site, where you could type in Markdown and see what it would look like, that might be quite a nice addition, because I imagine that is an obstacle for a lot of people.
I was thinking about something. I actually downloaded the purchases, the spreadsheet of purchase data, and the one piece of data that was missing, this is quite geeky, but the minimum price and the recommended price.
B: I would like to be able to know what they were when someone bought, because I’ve actually done quite a few experiments in seeing what happens to buyer behavior, depending on the pricing. So if you’ve got a minimum price and a recommended price, and they’re different, what do people do?
E: Ok, that’s great, that’s actually something we’d given a little bit of thought to, and it’s validating to hear that authors out there are actually thinking about that.
B: I think pricing is really interesting, actually. One of the reasons I used Leanpub as well was to learn about ebook publishing. Because I’m also dealing with journalists who are looking at this themselves. And that kind of, the idea of having a minimum price and a recommended price is a really interesting dynamic. For example, I found, for a while I made them both the same, and I noticed people almost never paid more than the recommended price, when it was the minimum. But if they are different, people do spend more than the recommended price.
E: That’s very interesting.
B: And obviously a lot of them spend the minimum. And when I did look at those figures, it averaged out still at the recommended price, but it was interesting that people would pay more, as well as less, when there’s an option.
E: Have you done any special one-day promotions, or anything like that? Like, where you lower the price or give out coupons?
B: No. I may well do that in the future. What I’ve really done is more of a gradual price-increasing. I think for the first week I said it was $4.99, so anyone who buys it in the first week, when there’s only one chapter, it was $4.99, and then it went up to $9.99, and it would increase as more chapters were added. Even though obviously everyone got all the chapters in the end. So it was kind of me saying, “I know that you’re buying something, and you don’t know yet what it’s going to be, so I’m reducing the price to recognize that”, and also recognize that the people who buy it at the start are quite important.
So I’ve not really done any price promotions in terms of reductions or anything like that. I might look at that in the future. At the moment sales are pretty constant, and have been throughout the thirteen months or so. But I may look at it in the future.
E: Well, that’s about all the questions I had. Is there anything you’d like to add?
B: No. I think you covered it all. Quite a wide range of ground I think we’ve covered there.
E: Well, thanks for a great interview, and for being on the Lean Publishing Podcast, and for being a Leanpub author!
B: And thank you as well, as I said it’s very educational.