Patrick Kua

Len Epp: Hi, I’m Len Epp from Leanpub, and in this Lean Publishing Podcast, I’ll be interviewing Patrick Kua.

Patrick works mainly as a technical leader and agile coach for ThoughtWorks, bridging the technical and non-technical realms, leading teams and writing software with a particular focus on helping teams improve through the practice of retrospectives. He blogs on various topics at THEKUA.COM@WORK, and you can follow him at @patkua on Twitter. Patrick often speaks at conferences, and you can find a number of his talks on YouTube, including The Geek’s Guide to Leading Teams, where he discusses the idea that the most challenging aspects to software development are always the people issues.

Patrick is the author of the Leanpub books, The Retrospective Handbook: A Guide for Agile Teams and Talking with Tech Leads: From Novices to Practitioners. The Retrospective Handbook draws on Patrick’s 10 years of experience running retrospectives to unlock their potential for your team, while Talking with Tech Leads lets you discover how more than 35 tech leads have learned to balance the technical and non-technical worlds.

In this interview, we’re going to talk about Patrick’s professional interests, his books, his experiences using Leanpub, and ways we can improve Leanpub for him and other authors. So thank you, Patrick, for being on the Lean Publishing Podcast.

Patrick Kua: Thanks for having me.

E: I usually like to start these interviews by asking people for their origin story. I was wondering if you could tell me how you first became interested in becoming a developer, and eventually became a tech lead?

K: Sure, absolutely. I guess I count myself lucky, because I fell into the industry relatively early. I mean, I’d played around with some Commodore 64-type stuff as a kid, but I never really got into the whole geek developer from very young. And then when I was in high school, we got introduced to sort of computers pretty early on. I had a really great teacher who was teaching us about software development, and I really got sort of, I guess my mind blown around the stuff that you could create from nothing. We were asked to develop a sort of film rental sort of system, using - back then - Microsoft Access, of all things – things I would never, ever recommend right now, and I kind of fell in love with the things that you could actually create.

That led onto a natural path of me wanting to study more about this at university, and I did a double degree in commerce and I.T. Double degrees are really popular in Australia, which is where I studied. So I ended up doing a commerce and I.T. degree - partly interested in the business side, as well as the technical side. So it’s already an interesting sort of balance out.

After university I went to start work for Flight Centre, whose headquarters are in Brisbane, and then joined shortly after, Oracle - who had an R&D center in Brisbane. There I was really lucky, because the team that I was working with, they were very early adopters of extreme programming. This is in the very early days of XP, which was - I guess - the early 2000’s, where we were playing around with the first continuous integration server, CruiseControl, which was actually built off of CVS, where we were introducing unit testing and JUnit.

For me, it was just such a really interesting contrast, because Oracle is kind of a behemoth, and there’s probably a lot of very traditional software practices - and we were a very small part of it, trying to change the way that we were doing software. This kind of naturally led on to where I ended up joining ThoughtWorks, whose Chief Scientist is Martin Fowler, one of the agile manifesto signatories. I’ve traveled around, and worked with ThoughtWorks for quite some time now. I think this year is actually the 11th year with ThoughtWorks.

E: That’s a long time. And have you been working with them in London the whole time?

K: I started with them in Brisbane, in Australia. We actually have, I think now - four offices. And, yeah, I transferred over to London - I think it was about 2005, where I’ve been based in London for probably the last 9 years. But being a global consultancy, we ended up - we were pretty good at actually shipping people around the world if they want, because it keeps our culture very consistent. It’s a way for us to sort of build our own network, and get exposed to new sort of experiences.

So, I’ve spent the tail end of winter in Calgary once - and summer, which was a beautiful city. I’ve spent about four months in India, amazing country - sub-continent. And I’ve spent, I don’t know? Probably on and off, probably a whole year in total with ThoughtWorks in Germany.

E: I read a blog post on your website about your year in Germany, and becoming fluent in a year. It was quite impressive, I thought, and I was wondering if you could maybe say a little bit about how you approached that year long project, and what the results were?

K: Absolutely. I guess I was kind of lucky in that - with ThoughtWorks, after 10 years, we have a sabbatical period. So, you get paid the equivalent of about three months’ worth of time, where you can choose what you want to do. When you’re presented with an opportunity to take three months off of work and do what you like - I was kind of like, “Oh, it’s really tempting. I never took that gap year that a lot of people take after university, maybe this is the time to do it?” So I actually had the year off, which was a fantastic opportunity, and I have so many great memories from that time.

I think, if you live in Europe and you come from an English-speaking background - like Australia or the U.S - it’s amazing how many people that you meet in Europe who are at least bilingual, if not more. For me it was kind of, like I don’t know - I would have always loved to learn another language fluently, and so for me this was a really great opportunity to do that.

Because I’d spent a little bit of time there for work, I’d learned some German. But it was what you might use as a tourist. So, I could order a beer, I could order for the bill, I could read some things off menu. But it’s really far different from actually living there and being there. So I made the choice to live in Berlin, which, to be honest, is not the best place to learn German. It’s so heavy with so many tourists, and I think Germany’s level of English is actually pretty high, so you can get away with actually not speaking any German if you really don’t want to. I know a few expat people who lived there for maybe three or four years, and their level of German is still sort of rudimentary, I guess tourist-level.

But knowing this helped me prepare. Because my motto going in was, it would be very easy to fall into this expat community and only speak English. So I really forced myself to sort of to pretend that I was in a small, little German village somewhere, where I was surrounded by only German speaking people. All the opportunities for the first, at least, three months, I would make sure that I’d be forced to speak German.

I had a really great flatmate, who knew that I was learning German, and was patient enough with me to only speak German to me - even though it was a very tough few weeks when I first moved in, and couldn’t communicate. I kind of felt like, I could now imagine how a baby would feel. In that, a baby who’s trying to get across a message - can’t, because they just don’t have the vocabulary, the words. And just that frustration, it’s phenomenal.

I think I remember breaking down one evening in dark winter in Berlin, saying, “Can’t we just speak English? I’ve just got to get this out of my system.” And my flatmate was actually really good with me, and said, “Nein!” in German. That was exactly what I needed, was the soft hand approach, but really firm that I was going to keep doing this.

I signed up for immersive German lessons, which was really important. So, I think we did classes from about 8:30 to about 1:00, every day. And then normally there would be homework and field trips afterwards. I think that really helps provide structure, and a really good learning ground to get the basics.

I think one of the benefits of actually going through an experience like that is, you also meet people at a similar level. So I think everyone’s patient with each other, because they understand, they’re also learning - and you’re just as patient giving other people space, because you know that they know the words in English, but you’re struggling to find and remember what the words are. That was really important.

I definitely recommend people get a tandem partner. There’s a concept that’s really popular in Germany - I don’t know how that translates and how popular that is overseas as well. But it’s basically somebody who’s learning the language that you’d like to learn, or who’s native in that language - and who’d like to learn English, or another language that you can speak. We would meet maybe once a week, for a couple of hours, and we would speak an hour of German, and then we’d switch to an hour of English, so that we both had time to practice. It was really helpful, because the language school that I went to helped connect us around personal interests. We spent a lot of time visiting different areas of Berlin, going to different events. It was just a really great way of meeting a new friend, having an opportunity to practice and focus, and get better at that.

E: It sounds like a really interesting idea, also because usually the teacher-student relationship, especially with language, has this kind of dominance on one side, and humiliation on the other. So, actually combining two people who each know that the other one is fluent in the language that they’re trying to learn, really balances that out.

K: Yeah absolutely. That’s such a key thing, and I think you hit on a point which is - I think there is that different relationship with a teacher and the students.

I guess another tip that I would give people is: it’s really worthwhile, if you’re going to learn something, to learn from lots of different people. It’s interesting, because everyone has their own bias about preferred learning style. And I think teachers have their sort of bias about how they believe that people learn best. It doesn’t always work.

So, I think we went through about four different teachers, over the course of three months. Just because of the level shifts, and different timings of classes. It was interesting for me to watch how one style of teacher didn’t really work for me, but it worked for other people. And then, another teacher who worked for me, didn’t really work for other people in that class. That’s where I think it’s really useful, to get exposed to lots of different teachers.

It’s something that I picked up from software development as well. Which is, you have so many different people who you can learn from, and each one has something very different to teach you. I think that’s a really important thing - when you’re progressing in any skill development, is that you get exposed to lots of different ideas. You may not necessarily take away everything that that person has to offer, but it does give you a different perspective.

E: Yeah, that reminds me of a line from your website where you’re talking about programming, but obviously it can apply to other areas as well - where you say that you believe strongly in self-empowered teams and individuals, having seen many talented people beaten down by the systems that they work in. The idea of empowerment, but also the element of individuality that’s inevitable in empowered teams, is also very important.

I was wondering if that came from - that particular feeling - maybe came from a personal experience you had with being beaten down in a big system, or was it something that you were fortunate to avoid yourself and just saw in other people?

K: I guess I consider myself pretty lucky, because I haven’t worked in a - I mean, I’ve worked with Oracle, but not in a large system where I felt beaten down, at least for a long time. I think that’s maybe one of the benefits of a consultant, is that you see lots of different environments.

What I really love is actually helping people get over these systemic things that have beaten the motivation out of people, where they’re just getting on with their tasks that they’ve been told to do, and helping them connect with the things that they really like. I’m a big believer in sort of Dan Pink’s Drive book - autonomy, mastery, and purpose. In that, if you can give people a reason to engage with the work that they do, and they can develop themselves at the same time, and they have an ability to choose the things that they think will help get them to that purpose – everyone benefits. Like, the people who are working on that sort of tasks are a lot happier - I think the job will be done a lot better.

And so, whoever that is for - be it the end customers or the business, everyone wins. I just wish that more people had that approach of, how can everyone find the solutions that work well for everyone? It won’t always be optimal, but trying to at least have that conversation with people around - “What drives you? What are you interested in? And how can we make that work for you in this environment?” I guess that’s what led me to be a very independent person, which I think helped me when I was learning my German language skills, is that I was just doing things that I felt interested me, that helped me - and then I’d find people to help me keep engaged, so I still had that purpose.

E: Yeah, it’s really interesting that you brought up the idea of a system. I’ve noticed this in your writing in a couple of places where you talk about why expert developers can sometimes make the worst tech leads. You talk about how someone can be extremely talented at a certain kind of system, like programming, but then the system that sits on top of it, where, what are you doing, and how does it fit into the larger scope of things, and the other people that you’re working with? Those talents are not - it’s not that they’re necessarily incompatible, but they’re not the same. I was wondering if you could explain a little bit about that.

K: Yeah, it’s really funny actually, beause I’ve done a lot of reading around systems thinking. The Fifth Discipline is a really great book, as is - I think it’s Thinking in Systems, I forget which one it is. I think what’s really interesting is just trying to - I think we’re taught in a certain way - though the way our education system runs, about being very scientific about things, boiling things down to singular causes and effects.

I think systems thinking teaches you to see things in a different perspective. Depending on where you draw the boundaries of a system, there are different perspectives and therefore different motivators, and different characteristics. For me, I think, when we work in a software environment, you have the local system of either your software system that you’re building, or your team that’s working on that software system. But there’s the broader organizational system in which it’s working, and I think for me, one of the challenges working as consultant is, you’re often brought in to try to shake up a system. You want to create an empowered environment. So you want to carve out a safe space for people to experiment, to gain their autonomy and confidence - and also then, deliver.

There’s a bit of a challenge there, because what’s good for individuals doesn’t always work for the whole system. There’s a trick in synchronizing the rate of change. One interesting thing that I see a lot with teams that really adopt a fully extreme approach to agile development - and the organization is a very big, lumbering behemoth - is that people won’t necessarily change the system, or they won’t fight to get the surrounding system around them changed.

We have this saying which is, “Change the system or change the system.” What that means is that people will get frustrated without any change in their own system. So they’ll choose to move to another company. They change their environment, and that’s another way of them changing the system. Which I think definitely is a consequence for companies that are thinking about moving in a very different cultural way from what their current system is. But at the same time for me, it’s - you’ve helped people connect with purpose and something good for them as well. It’s quite hard, because we’re trying to also appease the people that we work with, as well as our customers who may be quite different - or in different circles.

E: That’s really interesting. I really like the observation about how we often try to take reductive causal frameworks, and then apply them to the complexities of systems, like human systems for example, that they don’t quite apply to.

I’ve got a joke which is, it was an apple that fell on Newton’s head, not a person. If an apple had fallen on his head, then he would have the insight that he did about how these simple formulas can explain the motion of the planets very far away. But if a person had fallen on his head - he would have freaked out, and jumped up and had all sorts of different thoughts - none of which had anything to do with this simple explanation of how things move in the universe. And that, actually since the Enlightenment, there’s been a problem where people have often tried to take the successes of the sciences, and then apply them to areas where the systems are actually different. There was actually a German philosopher, Husserl, who wrote a book about that - over 100 years ago [sic], so it’s a longstanding problem.

But I had a specific question. Your book, “The Retrospective Handbook,” is about retrospectives. This is in particular a way of managing people within these systems, and helping them to improve and maybe find their own place. I was wondering if you could maybe explain to people listening what a retrospective is, and why it’s so important?

K: Absolutely. What’s really interesting is that I think in most businesses, the pace of change is really rapid, and people expect things to happen. I think often there is the assumption that everyone is doing the best that they can, which I also believe. People are given goals, and they’re all working towards that.

But it’s hard to improve in that sort of work environment. And for me, the retrospective is sort of a time out from your normal day-to-day work environment of - all these things going on, all these priorities. All these things that are changing. And creating a bit of space to actually reflect. For me, the power of retrospective is this meeting where you bring people who are working perhaps on the same sort of goal together, where they can talk in a bit of a safe environment, to talk about what things that they could change or improve.

For me, the key outcome of a retrospective is really about the small actions that they might take. To slightly experiment or to improve something. I don’t think you necessarily need to change everything. But for me, the idea of a retrospective is about change, and that safe environment to talk about what’s gone less well in our environment, and what are the things that have gone well - and maybe can we amplify them? It creates that space for a whole team or a group of people to come together, which, a lot of businesses don’t really make time for. And so, they never really take the opportunity to understand what improvements they might have.

E: Is a retrospective something that happens continually throughout - let’s say a project? Or is it something you do at the end? You know there’s this terrible term “post-mortem”. It just sounds like it’s different from that, right?

K: Yeah absolutely, and you’re spot on - a post-mortem is after everything is dead. So you can’t really do anything to change it. The idea for a retrospective is to keep these short, to do them regularly. I encourage groups of people to do them maybe once a week or maybe once a fortnight, and to try to take small steps towards something.

For me, I’m a big believer that small steps can lead to huge change. Therefore, the small experiments that people can take give people an opportunity to review them the following week - and maybe change them back if they didn’t go so well. I think it’s a really easy way to introduce new things, because it’s low risk for people, right? It’s like, if they know that there’s a way back, people are more likely to sort of try something out. So yeah, you’re spot on in that it is a bit more of a regular sort of cadence of how you run retrospectives.

E: You write in your book about the retrospective prime directive. I was particularly interested in how important it was, in your experience, that actually this is read out explicitly at the beginning of a retrospective - or when people are being introduced to the retrospective process. And I’d like to actually read the sentence.

K: Sure.

E: And have you explain why in your experience it is so important that it’s read out. So here it goes.

K: Sure.

E: The retrospective prime directive: “Regardless of what we discover, we understand and truly believe that everyone did the best job he or she could, given what was known at the time, his or her skills and abilities, the resources available, and the situation at hand.”

K: Yeah, so firstly, I won’t take credit for this, because I’m standing on the shoulder of giants. This actually comes from Norm Kerth’s original book about project retrospectives, which were also considered more at the end of a project, that you might do something. For me, I think this statement is a really good example of that difference between a retrospective and a postmortem.

For me, I think this statement is actually really important, particularly for people who’ve never done a retrospective before. I’ve worked with teams that have been working a couple of years, and they’ve been doing whatever they do. Nobody’s ever really asked them about what could they change? How could they improve things? And that ability to influence their working environment.

For me, this is a really great way of sort of introducing the concept of safety. Of, we aren’t here in a retrospective to point fingers and hang people up to dry. We are here to really believe that everyone was doing the best that they could, and to put our mind on the system that we can influence, rather than pointing fingers and blaming people, because that doesn’t really improve the system.

I think for people who’ve been part of retrospectives for a while, they understand this. But I think, particularly for new organizations, what I would say is - maybe more conservative sort of teams or cultures, where change doesn’t happen a lot, and they’re not used to it - I think it’s a really powerful statement to open a retrospective and create that safety. My rule of thumb is that people should try to use this the first time, if they’ve never run a retrospective with their team before.

E: It’s interesting, you talk about safety and in your book, you also talk about bias. I was wondering if that’s related to the importance that you place on independent facilitators for retrospectives?

K: Yeah absolutely. I have a very strong memory of going to a client where a retrospective was run by a very, I would probably describe, controlling project manager. I think it was my second week with the client, and so I sat in the retrospective - I didn’t have that much to offer, because it was the second week. I just watched how that project manager would take the white board marker, point at somebody and say, “Okay, what went well last week?” Point at another person and say, “What didn’t go so well?” And it was really interesting, because the responses from those people were pretty much one word answers. There was no elaboration about story. There was no context given to those points. For me, it was such a contrast when I’ve seen independent facilitators who don’t have any other agenda - who have no power relationship with the participants. It does really create a good, safe environment that people can talk a lot better in. So I think it’s a really powerful concept to have an independent facilitator. I think it really increases safety.

E: Yeah, it’s really interesting too how when you’re talking about, in Talking With Tech Leads - your second Leanpub book I think? - you write that the experience you gained as a developer does not prepare you for the responsibilities of a role of a tech lead. Even though it’s important that as a tech lead - which you define as someone who is actually still coding usually, right? And participating in the creation of the code for the project, but also now has been moved into a space in where they’re steering the ship, and all the skills that you’re talking about now - not pointing and kind of demanding one word answers. It’s effectively demanding one word answers.

I was wondering if you could explain a little bit about your experience interviewing all the people for Talking With Tech Leads - what insights you might have about people who’ve newly moved into that role, or might be a little bit worried about being pushed into that role, and how they can shift from being a developer to being a tech lead?

K: Absolutely. It was really interesting talking to so many different people, because when I set out to do the book, I didn’t really have an idea what responses I might get. The book was intentionally structured so that I posed the same sorts of questions to people. And what emerged was interesting things that people had different - I guess when they had different perspectives on or different times of their career. As they were maybe, earlier in their phase or later in their phase, about what areas that they focused on. So there were some natural themes that developed.

I think for people who are very new to it, who I call the sort of novices, there is a huge shift from what you do as a developer, to suddenly leading a team. As a developer, you’re normally very focused on how well your code is structured and how nicely everything runs. You don’t really need to worry too much about sort of the people in your team - that’s somebody else’s problems, and you don’t necessarily need to worry about where work comes from and managing business people. And then when you’re suddenly thrust into this role of being a tech lead, you’re trying to also write a little bit of code, but at the same time you’re pulled in lots of different directions.

That’s quite a natural state, and I think my first word of advice for people in that role is to not panic, because it can be really overwhelming. Time management skills are quite useful, trying to prioritize through some of this stuff. I think also the big difference, a big jump is - as a developer, you feel like you have a lot of control over the code that you’ll end up writing. And I think as a lead, it’s difficult to give up that control.

So, back to that project manager who’s really restricting what people say: letting go is really hard. As a developer, you’re really opinionated about how something might be written. And as a lead, you still have to have some guiding principles - you have some sort of say. But you have to be okay that the method or the structure won’t be written exactly how you would’ve done it. In fact, it may actually be better for it, because somebody has a different approach from you for actually solving things. So, there’s a big shift mentally I think, that developers have when they move into this role of suddenly realizing that it’s okay that problems get solved - but in a different way, and that’s okay.

E: I was just fascinated reading the stories from different people about their different experiences. It was really interesting how in the book, you ask people the same set of questions, right? But they came back with these unsolicited themes that grouped together.

K: Yeah, that was very fascinating.

E: I was wondering actually, on the subject of publishing - you actually turned Talking With Tech Leads into a print book.

K: Yeah.

E: And it’s for sale in 3 different regions on Amazon.

K: Yes.

E: I was wondering if you could explain a little bit about that experience, of turning it into a print book from an ebook.

K: Absolutely. I’ve been a fairly early adopter of Leanpub. I think it was actually with The Retrospective Handbook, that I was trying to turn into a printed book as well. I was asking after a print-ready, PDF version. I used the CreateSpace publishing platform, which is sort of a part of Amazon. And then, they self-publish on demand a printed book.

But they require a whole bunch of preparation, like a PDF version that is printable. And it’s quite an interesting process about moving from an ebook format to a PDF that is print ready. Because there’s lots of little formatting differences that make a big difference when you’re in a print version. A simple example is the alternating page numbers from left to right. You want them to be on the outside of the book, which, when you’re on a single screen, it doesn’t really matter. The same thing with margins is that, when it’s printed, you want the margins that are going to be bound together a little bit deeper, because you want it to look equal - so when somebody opens the book, it looks nice.

Leanpub has been really awesome at actually picking up feedback. I was generating lots of feature requests for understanding what a print-ready PDF could look like. And then I would do a test run with Amazon and give feedback, saying, “Okay, well the test copy, now that I see it in real life, needs some adjustment.” I’ve been a huge fan of Leanpub from the very beginning, because it was a really great example of lean flow in action. Which was - I was creating a demand, and your platform was responding to that as I was getting feedback from it. I wasn’t quite sure what I really wanted either. It was really useful to have those really fast feedback cycles.

E: Yeah I remember seeing all the interaction between you and Scott, one of Leanub’s co-founders. During that process, you really - one of the reasons I’m asking you is because you’ve really pulled a lot of that out of us, and we were so glad when someone arrived on the scene and was like, “I want to have good output to make a good print book.” Are you happy with where it is right now? The print output and everything kind of works more or less the way you need it to, to get it on Amazon and things like that?

K: Yeah absolutely. I mean, what was really interesting was that - I guess a comparison between the first draft of The Retrospective Handbook, and then preparing for the Talking With Tech Leads print version. I don’t think there was any feature request that I had to change at all. In fact, I think for me, the difficult part was actually the wraparound cover - which is something that you don’t necessarily need to worry about in an ebook world. It is something that is exclusively to do with print layout. You need to think about what goes on the back, what goes on the spine, and try to get the formatting right based on the thickness of the book - it’s kind of one of those tricky - you have to print it, really, to see what the end result is. But from a platform perspective, it was exactly what I needed - and I didn’t have to do any extra changes to it.

E: Oh that’s great to hear, I’m really glad. I’m really glad that it’s at that stage. I was wondering, I read that you used a copy editor, and it might have been the same person for both of your books. Can you say a little bit about what that experience was like, and how you found a copy editor?

K: Yeah absolutely. I found a copy editor through a colleague of mine actually. I think for one of the very early drafts of The Retrospective Handbook, I’d asked some people to give me some feedback on it. I think one of the early bits of feedback is that, “You’re really too close to your writing.” I think this is probably one of the dangers of self-publishing: you decide to go down the, “I can take care of everything” route. “I can do all the grammar and checking.” But actually that’s not my strength, and that’s - I would actually say that’s probably one of my weaknesses, is that, I tend to speed read, so I tend to skip over things that other people would notice.

I think there’s a lot of value in working with an editor. So I worked with Angela from Virtual Editor, Angela Potts. She was really - she’s based in Cambridge. I guess one of the really great things is that, we still physically haven’t actually met. We actually met over email, and then we did some Skype sessions - and so she works very virtually as well. I think the Leanpub process was actually really useful, because I could just share simple text files with her. With enough simple syntax, she understood how to sort of start editing some of those files and we could actually just simply trigger draft copies really rapidly. That was a really useful process, of actually sharing the book and getting fast iterations over the copy.

E: So did you actually share, like give her access to the Dropbox folder that you would have been working in and direct access to the files?

K: Yeah absolutely.

E: Okay.

K: Yeah, and that worked really well. For the first book, it was just a bit more copy editing. I already had the structure in place. And then our relationship evolved a little bit more, and she helped me with a little bit more of the structure for the second book - how to make it a bit more interesting, rather than just a whole series of interviews.

E: I noticed that both of your books have really good covers, and I was wondering if you made them yourself, or did you use an external service to do that? That’s one of the questions we get all the time from people, because a great cover really does make a big difference, particularly in confidence that people have in a book, especially if it’s self-published, or maybe even in-progress.

K: Yeah, it’s a great question. I designed them myself. I wasn’t sure about this actually to begin with. I was also thinking about going out to, I guess the market - what some other authors have done about trying to get somebody to pitch different ideas towards a cover. But for me, it was actually kind of like a fun, creative experiment. My goal was to create something that was a little bit eye-inducing. It could be a lot better, but it’s also the skills that I had.

I ended up studying actually, probably about - there’s a couple of websites that show really great book cover design. I looked around at I think a site that showed like 50 or 100 of them, and then I was looking at some of those things, thinking about what sort of themes and colors and - I guess - patterns that I might take, that I could reproduce in open source tools. So I used GIMP for the actual file creation.

E: Okay.

K: And then I just ended up coming up with a couple of different designs. I also did the whole sort of lean testing as well. I actually generated quite a lot of different variants, and then I tested it against some of my colleagues and close friends to see, “Do you like A or B better? What do you think about this color? How’s these patterns?” I ended up settling on the style that you see on The Retrospective Handbook.

Then I wanted to continue that theme with the second book. That was a little bit more interesting, because I was actually - it was really important for me in Talking With Tech Leads, to get a diverse sample of people in there. So what might be surprising, which I haven’t really written a lot about, is that, I’ve managed to probably get about a third of the people in the book - I think - to be female tech leads. It’s really hard to get female developers, let alone people in leadership roles. And so, when I designed the cover, I also was very conscious about making sure it wasn’t a male or a female looking tech lead. I’ve tried a sort of gender-neutral-looking icon, which is what my design intentions were. I don’t know if it’s come through, but yeah - I guess that was the sort of - So, have some sort of theme that comes along from the first book, but have something that’s representative about that sort of topic, yeah.

E: That’s really interesting. I mean, obviously you’re very hands on with the scope of the entire project now. I was wondering if you’d ever - for both books, I was wondering if you’d ever considered going with a conventional publishing process or conventional publisher to begin with? Or was that just something that never occurred to you, and publishing it yourself just seemed like the way to go?

K: I think for the first book, I had thought about going to a publisher. But I was just really surprised at how rapidly it had gone through. I mean there’s - I know a lot of people who’ve written books for the Prags and O’Reilly and things like that, and I think there’s trade-offs. I wanted to go down the whole self-publishing route to see how easy or hard it was going to be. And I’ve been really happy with the reach of the platform and the ease with which Leanpub makes it easy for people to buy and distribute the books all over the world.

I mean, I might go through that process, but for me I think the editor is probably the biggest value that one of those traditional platforms can bring - and that they have a strong network. And probably around the consistent theme. So if it fits in with an existing series like the Martin Fowler signature series, or something like that, that probably makes sense. But I’ve been very happy with the whole self-publishing route.

I guess the only thing that I’m thinking about, is how better to market the book. But I think that’s part of the self-publishing process, right? Which is, what can you do to get the book out there?

E: Yeah definitely, definitely. And that’s kind of the - writing’s the hard part, but so’s the marketing.

K: Absolutely.

E: I was wondering actually, I guess my last question would be - if there was one dream feature that you could have us build for you, what would it be? If there was one thing that was either missing or that you now - looking back, or even looking forward - which we could give to you as part of our process, what might that be?

K: It’s a really great question. And to be honest, I don’t have a good answer. I mean, one of the things I really like about you guys is that - if I want something, I feel that I can post something on the forums and get a response, and have somebody either say, “It’s on the road map,” or, “We’ll work on it.” I think that’s such an amazing opportunity of the platform. And so, I can’t really think of another feature that I’d really like - otherwise I would have asked for it by now.

E: Well thanks - if you ever do, please come back and ask again. A big part of our process is having people approach us and tell us what they need, and then us delivering it. I mean after - obviously with deliberation!

So anyway, Patrick, thank you very much for your time and for being on the Lean Publishing Podcast, and for being a Leanpub author.

K: Thank you for having me.

This interview has been edited for conciseness and clarity.

Leanpub's Year In Review: Looking Back on 2015 And Looking Forward To 2016

by Len Epp

published Dec 22, 2015

In this post I’m going to highlight some of the improvements we made to Leanpub for our authors and readers in 2015, and discuss briefly what our main goals are for 2016.

Being a bootstrapped startup is a big and exciting challenge for the people involved, but not always in obvious ways. Our development team does a ton of work building new features and products all the time (like our “Bring Your Own Book” feature and our iOS app), but as with any startup, there’s also a ton of work happening in the background that no one sees. And there are of course the usual ups and downs, the best ups being when you see that you’ve really helped people do amazing things, the worst downs when you feel like you’ve failed them in some way.

This is our chance to recap what we’ve done this year, and speak broadly about what we’re planning to do next year.

The Top Books On Leanpub In 2015

We’ve already posted about the most important thing we focus on at Leanpub: the success of our authors. We’re proud to say that by this measure, 2015 was our best year ever. 2015 saw many new books from many new authors on a wide range of topics.

Leanpub is still dominated by technical books written by technical authors, but authors and academics with diverse interests are constantly kicking the tires. Hopefully we’ll soon be attracting more fiction authors, in particular authors who are writing serial fiction. Peter likes to talk about Dickens, Dostoyevsky and Mary Elizabeth Braddon a lot, but Leanpub hasn’t caught up yet to our vision for serial publishing.

If you look at either the list of our top ten bestsellers or the list of our top ten most popular books, you can see that Data Science was by far the most popular book category of the year.

New Leanpub Features and Initiatives In 2015

We’re always building new features and launching new initiatives at Leanpub, often in response to requests from authors and readers who’ve thought up new ways for us to improve. Of course, this doesn’t mean we just do what we’re told, but we do work hard to internalize everything we hear so every decision we make is informed by the customer development process.

Packages

One of the most important new features we built this year was Packages. This lets authors sell videos, code samples and other digital content along with their books. Given the huge and growing popularity of online video courses as a form of instruction and education, this feature has been used successfully by a number of Leanpub authors.

In fact, packages accounted for 10% of our sales by revenue in the last year. Perhaps the most notable example of a successful use of packages is Roger D. Peng’s R Programming for Data Science.

If you’ve written a programming or a technical book and you have the time and the inclination, we definitely recommended producing an instructional video to sell along with your book in a package.

People love videos, so even if you don’t have the time to make a lengthy video to accompany your book, you can still make a short book trailer that people can watch from your book’s landing page. The one on this page is a great example.

Leanpub Podcast Reboot

We’re really glad we rebooted the Leanpub Podcast in June 2015, since it’s so much fun hearing directly from authors about their work. I had a great time interviewing a number of successful Leanpub authors, including Roger D. Peng, W. Jason Gilmore, Ryan Bigg, Jeff Leek, Paul M. Jones, Brian Caffo, and Thomas Davis.

In the hopper awaiting transcription are even more interviews, with Patrick Kua (coming very soon!), Jeff Geerling, Emmanuel Nataf, and Claire Bowern.

Shopping Cart

Yes, it’s true: at the beginning of 2015 we still didn’t have a shopping cart. We received surprisingly few requests for one, actually, but obviously we were really glad when we finally deployed this.

Our Brief Experiment With An Affiliate Program

We experimented for a couple of months with an affiliate program, whereby authors who signed up for the program could share their royalties with “affiliates” who posted links to the authors’ books that generated sales. Eventually we shut the program down for a number of reasons; basically, the whole thing just wasn’t very “Leanpub”. We may try a version of this program again in the future, but for now we’re focusing our attention elsewhere.

Buying And Distributing Multiple Copies of Ebooks on Leanpub

Something we should surface more openly is the ability to buy multiple copies of ebooks on Leanpub.

If you’re buying on behalf of a team, we’ve built a great feature that lets you distribute individual copies of Leanpub books to the individual members of your team, using just their email addresses. That means they don’t just get a copy of the book: they also get a Leanpub account where they can access the latest version of the book any time, hassle-free.

Bring Your Own Book

We designed a workflow that lets you upload your own book files to the Leanpub bookstore, rather than using the Leanpub writing workflow.

A good way of describing one of our main goals at Leanpub is that we want to give authors a magical typewriter: a great way to write that suits individual preferences and, with the click of a single button, lets authors publish their work in multiple ebook formats and put them up for sale on the web, while simultaneously distributing an updated version of their books to all their existing readers.

In pursuit of the magical typewriter, the first thing Leanpub did was build a writing workflow that let people write on their own computers, using their favorite plain text editors, and share their work with Leanpub through Dropbox. This eliminated the common step in the publishing workflow of uploading or sending files in a separate, manual sharing process, and then converting the text from one format to another. It’s an important feature of the magical typewriter.

Leanpub authors write their books in a simple plain text markup format we call Leanpub Flavored Markdown. (In Q1 2016 we will launch the successor to Leanpub Flavoured Markdown, Markua, while still supporting Leanpub Flavored Markdown for the foreseeable future.) Essentially, this is the magic workflow that lets you write in plain text while simultaneously, efficiently, and easily doing the work required to automatically generate good-looking ebook files in multiple formats with one click, using the Leanpub book generators. I think of it as punctuation for publishing in the digital age.

Over time, we introduced integration with GitHub and Bitbucket in addition to Dropbox. We also built our own in-browser editor, so authors wouldn’t even have to share a folder with us.

Eventually we realized that in order to accomodate the needs of even more authors (especially those with already completed books!), we needed to accept that not everyone would want to write with any markup format, no matter how simple or how superior in a technical sense to a more conventional way of writing. We strongly believe that more and more people will switch to writing in plain text over time. If people could at one time switch from pens to keyboards, they can certainly switch to writing in plain text, which is in so many ways better than writing in, for example, Microsoft Word. But conventions like these can take a long time to change, and we knew we needed to adapt.

Also, we knew that there were authors coming to Leanpub in order to take advantage of our in-progress publishing workflow and features, but who wanted to have much more control over formatting details than a plain text markup format can offer. This is especially likely if the author has finished writing the book, which is the proper time to start focusing on detailed formatting, if it’s important to the project. (One of our basic principles is that for almost all writing projects, formatting is procrastination unless the writing is already done. This is especially true if people are reading on phones or e-readers where they can change the text size, screen orientation, font, text and background color, etc.)

Finally, by restricting authors to using the Leanpub workflow, we were excluding everyone who had already made an ebook using some other process. We encourage Leanpub authors to try other platforms, but at the same time, of course we want authors who are already selling books on other platforms to also give Leanpub a try.

So, we added a “Bring Your Own Book” workflow, which lets anyone easily upload their own book files onto Leanpub and distribute their book through our bookstore. This has attracted some amazing projects and authors, including some where authors had already completed books but couldn’t find publishers for them, like these.

A New Role for Len at Leanpub

I was granted cofounder status at Leanpub this year, joining my co-founders Peter Armstrong and Scott Patten.

Looking Ahead To 2016

We’ve got a lot of things planned for 2016.

First, Markua will launch in Leanpub near the end of Q1. Markua will give authors an even simpler and more powerful plain-text writing experience: it’s the result of over a year of thinking about how Leanpub Flavoured Markdown should evolve. You can read the spec here right now – it’s being published on Leanpub while it is in-progress: we believe in Lean Publishing. Markua isn’t just for Leanpub, by the way–we’re shipping an open source version too. See http://markuapad.com/ for a demo and https://github.com/markuadoc for the various repositories.

Second, if you’ve had problems syncing your Leanpub manuscript via Dropbox in 2015, we’ll have a proper solution for that in 2016 as well. Stay tuned.

However, without revealing all the details, the biggest change to Leanpub in 2016 is going to be an expansion of our priorities. This will involve paying a lot more attention to the reader experience.

Since Leanpub was created by book authors for book authors, it’s always been our natural inclination to try to solve the hard writing- and publishing-related problems first. We’ve got more exciting developments for authors planned, but one thing we’ve always looked forward to spending more time on, is radically improving the experience for readers.

This doesn’t just mean improving the reading experience on our app and online (the current online versions of Leanpub books actually need quite a bit of work, we know), and adding reviews, which is something we plan to do soon. It means thinking really hard about how in-progress publishing should look from the reader’s perspective.

What should a library of unfinished books look like? What features should it have? How should readers be encouraged to interact with authors who are still working on their books? What’s the best platform we can build for delivering a coherent experience to readers, that helps better connect them with authors in the Leanpub spirit?

We’ve got lots of work to do 2016, and we have a clear idea of where we going. Peter likes to describe Leanpub as “Steam for books” – by the end of 2016 it should be a lot clearer why we think this way. Please follow us on Twitter and stay tuned for special announcements in the new year. We’re looking forward to showing you what we’ve come up with, and listening to your feedback whenever we release something new.

Thanks!

As always, if you have any suggestions for improvements or new features, please reach out to the entire Leanpub team at hello@leanpub.com.

Thanks to everyone in the Leanpub community, authors and readers alike, for making 2015 such a great year!

The Top Ten Most Popular Leanpub Books In 2015

by Len Epp

published Dec 16, 2015

Leanpub books are sold using a variable pricing sales model. This means that for each book, the author or publisher sets both a minimum and suggested price for the book, and the reader chooses what to pay.

This means that measuring a book’s success in the Leanpub marketplace is in some ways different from the conventional bestseller lists you see. For this reason, in the Leanpub bookstore we like to show two types of lists for the top-performing books: those that have earned the most sales revenue, or “Bestsellers”, and those that have earned the most readers, or “Popular Books”.

This distinction allows us to clearly distinguish success in terms of revenue from success in terms of the number of readers reached.

We’ve already posted our list of the top ten bestselling Leanpub books in 2015. Below is a list of the most popular books on Leanpub in 2015. Some of them are naturally enough also on the top ten bestsellers list, but some aren’t.

Perhaps the most interesting thing to note, for people interested in the ebook publishing market, is the fact that the top two books in both lists have free minimum prices.

There are even more data science books on this “Popular Books” list than there are on the “Bestsellers” list. That’s partly because so many of the data science books have a minimum price of free, which means the connection between number of readers and sales revenue is less direct than in other cases, where the minimum price of a book is not free.

At the same time, the dominance of this category of book on Leanpub is also explained by the explosive popularity of data science as a discipline. Check out the Leanpub Podcast interviews with Roger Peng, Jeff Leek and Brian Caffo if you’re interested in hearing experts talk about what data science is and why the discipline is growing so rapidly. You can also find full transcriptions of the podcasts on our blog.

If you’re interested in hearing about Malcolm Maclean’s journey onto our popular books list with https://leanpub.com/D3-Tips-and-Tricks: Interactive Data Visualization in a Web Browser, check out our interview with him here.

Thanks again to all our authors and the growing community of Leanpub readers for making 2015 such a great year!

Leanpub’s 2015 Top Ten Most Popular Books List

#10. Data Analysis for the Life Sciences by Rafael A Irizarry and Michael I Love

Title: Data Analysis for the Life Sciences
Authors: Rafael A Irizarry and Michael I Love
Description: Data analysis is now part of practically every research project in the life sciences. In this book we use data and computer code to teach the necessary statistical concepts and programming skills to become a data analyst. Instead of showing theory first and then applying it to toy examples, we start with actual applications and describe the theory as it becomes necessary to solve specific challenges. The book includes links to computer code that readers can use to follow along as they program.

Tweet to Rafael and Mike at @rafalab and @mikelove

#9. PHP: The “Right” Way: Your guide to PHP best practices, coding standards, and authoritative tutorials by Phil Sturgeon and Josh Lockhart

Title: PHP: The “Right” Way: Your guide to PHP best practices, coding standards, and authoritative tutorials
Authors: Phil Sturgeon and Josh Lockhart
Description: There’s a lot of outdated information on the Web that leads new PHP users astray, propagating bad practices and insecure code. PHP: The Right Way is an easy-to-read, quick reference for PHP popular coding standards, links to authoritative tutorials around the Web and what the contributors consider to be best practices at the present time.

Tweet to Phil and Josh at @philsturgeon and @codeguy

8. Report Writing for Data Science in R by Roger D. Peng

Title: Report Writing for Data Science in R
Author: Roger D. Peng
Description: This book teaches the fundamental concepts and tools behind reporting modern data analyses in a reproducible manner. As data analyses become increasingly complex, the need for clear and reproducible report writing is greater than ever. The material for this book was developed as part of the industry-leading Johns Hopkins Data Science Specialization.

Tweet to Roger at @rdpeng

7. D3 Tips and Tricks: Interactive Data Visualization in a Web Browser by Malcolm Maclean

Title: D3 Tips and Tricks: Interactive Data Visualization in a Web Browser
Author: Malcolm Maclean
Description: Over 600 pages of tips and tricks for using d3.js, one of the leading data visualization tools for the web. It’s aimed at getting you started and moving you forward. Includes over 50 downloadable code examples. You can download for FREE or donate to encourage further development if you wish :-).

Tweet to Malcolm at @d3noob

6. Regression Models for Data Science in R: A companion book for the Coursera Regression Models class by Brian Caffo

Title: Regression Models for Data Science in R: A companion book for the Coursera Regression Models class
Author: Brian Caffo
Description: This book gives a brief, but rigorous, treatment of regression models intended for practicing Data Scientists.

Tweet to Brian at @bcaffo

5. Statistical inference for data science: A companion to the Coursera Statistical Inference Course by Brian Caffo

Title: Statistical inference for data science: A companion to the Coursera Statistical Inference Course
Author: Brian Caffo
Description: This book gives a brief, but rigorous, treatment of statistical inference intended for practicing Data Scientists.

Tweet to Brian at @bcaffo

4. Exploratory Data Analysis with R by Roger D. Peng

Title: Exploratory Data Analysis with R
Author: Roger D. Peng
Description: This book teaches you to use R to effectively visualize and explore complex datasets. Exploratory data analysis is a key part of the data science process because it allows you to sharpen your question and refine your modeling strategies. This book is based on the industry-leading Johns Hopkins Data Science Specialization, the most widely subscribed data science training program ever created.

Tweet to Roger at @rdpeng

#3. The Art of Data Science: A Guide for Anyone Who Works with Data by Roger D. Peng and Elizabeth Matsui

Title: The Art of Data Science: A Guide for Anyone Who Works with Data Authors: Roger D. Peng and Elizabeth Matsui
Description: This book describes the process of analyzing data in simple and general terms. The authors have extensive experience both managing data analysts and conducting their own data analyses, and this book is a distillation of their experience in a format that is applicable to both practitioners and managers in data science.

Tweet to Roger and Elizabeth at @rdpeng and @eliza68

#2. The Elements of Data Analytic Style: A guide for people who want to analyze data by Jeff Leek

Title: The Elements of Data Analytic Style: A guide for people who want to analyze data.
Author: Jeff Leek
Description: A guide for people who want to analyze data.

Tweet to Jeff at @jtleek

1. R Programming for Data Science by Roger D. Peng

Title: R Programming for Data Science
Author: Roger D. Peng
Description: This book brings the fundamentals of R programming to you, using the same material developed as part of the industry-leading Johns Hopkins Data Science Specialization. The skills taught in this book will lay the foundation for you to begin your journey learning data science.

Tweet to Roger at @rdpeng

The Top Ten Bestselling Leanpub Books In 2015

by Len Epp

published Dec 15, 2015

2015 was Leanpub’s best year in a number of ways, but the most important thing to us was seeing so many great books published by both new and incumbent Leanpub authors, reaching tens of thousands of new readers.

Every successful book has its own story. This is perhaps true in a unique way for Leanpub books, given our in-progress pubishing and variable pricing sales models.

So before we get to the list, here are some of the highlights:

Jeff Geerling’s book, Ansible for DevOps: Server and configuration management for humans made $25,000 in sales before he finished writing it, as he details in his great post, $25K in book sales, and I’m almost ready to publish.

Axel Rauschmayer’s book, Exploring ES6: Upgrade to the next version of JavaScript, made it to #9 on the list of the top ten bestsellers while simultaneously being available for free online outside the Leanpub bookstore.

Roger D. Peng’s R Programming for Data Science became the first Leanpub book to reach over 50,000 readers.

Data Science books had a huge presence on Leanpub this year, thanks largely to the contributions of a number of US-based academics. This movement was led by four scientists in particular, and we’d like to offer them special thanks and congratulations for their achievements and generosity (many of their books are available for a free minimum price on Leanpub): Brian Caffo, Jeff Leek, Roger D. Peng, and Elizabeth Matsui. For interviews with Roger, Jeff, and Brian, check out the Leanpub Podcast (full transcriptions are available on our blog).

Leanpub’s 2015 Top Ten Bestseller List

#10. Ansible for DevOps: Server and configuration management for humans by Jeff Geerling

Title: Ansible for DevOps: Server and configuration management for humans
Author: Jeff Geerling
Description: Ansible is a simple, but powerful, server and configuration management tool. Learn to use Ansible effectively, whether you manage one server—or thousands.

Tweet to Jeff at @geerlingguy

#9. Exploring ES6: Upgrade to the next version of JavaScript by Axel Rauschmayer

Title: Exploring ES6: Upgrade to the next version of JavaScript
Author: Axel Rauschmayer
Description: An in-depth book on ECMAScript 6, for JavaScript programmers.

Tweet to Axel at @rauschma

#8. The Art of Data Science: A Guide for Anyone Who Works with Data by Roger D. Peng and Elizabeth Matsui

Tweet to Roger and Elizabeth at @rdpeng and @eliza68!

#7. Software Architecture for Developers: Technical leadership by coding, coaching, collaboration, architecture sketching and just enough up front design by Simon Brown

Title: Software Architecture for Developers: Technical leadership by coding, coaching, collaboration, architecture sketching and just enough up front design
Author: Simon Brown
Description: A developer-friendly, practical and pragmatic guide to lightweight software architecture, technical leadership and the balance with agility.

Tweet to Simon at @simonbrown

#6. Exploratory Data Analysis with R by Roger D. Peng

Title: Exploratory Data Analysis with R
Author: Roger D. Peng
Description: This book teaches you to use R to effectively visualize and explore complex datasets. Exploratory data analysis is a key part of the data science process because it allows you to sharpen your question and refine your modeling strategies. This book is based on the industry-leading Johns Hopkins Data Science Specialization, the most widely subscribed data science training program ever created.

Tweet to Roger at @rdpeng

#5. Build APIs You Won’t Hate: Everyone and their dog wants an API, so you should probably learn how to build them by Phil Sturgeon

Title: Build APIs You Won’t Hate: Everyone and their dog wants an API, so you should probably learn how to build them.
Author: Phil Sturgeon
Description: Tasked with building an API for your company but don’t have a clue where to start? Taken over an existing API and hate it? Built your own API and still hate it? This book is for you.

Tweet to Phil at @philsturgeon

#4. ember 101: Learn Ember.js with ember-cli by Adolfo Builes

Title: ember 101: Learn Ember.js with ember-cli.
Author: Adolfo Builes
Description: Everything you wish you had known while learning to write Ember.js applications.

Tweet to Adolfo at @abuiles

#3. MEAN Machine: A beginner’s practical guide to the JavaScript stack by Chris Sevilleja and Holly Lloyd

Title: MEAN Machine: A beginner’s practical guide to the JavaScript stack.
Authors: Chris Sevilleja and Holly Lloyd
Description: A beginner’s practical guide to the JavaScript stack.

Tweet to Chris and Holly at @sevilayha and @hollylawly

#2. The Elements of Data Analytic Style: A guide for people who want to analyze data by Jeff Leek

Title: The Elements of Data Analytic Style: A guide for people who want to analyze data.
Author: Jeff Leek
Description: A guide for people who want to analyze data.

Tweet to Jeff at @jtleek

1. R Programming for Data Science by Roger D. Peng

Tweet to Roger at @rdpeng

A Leanpub Interview With Malcolm Maclean

by Len Epp

published Dec 07, 2015

Recently over email I interviewed Malcom Maclean, one of the most popular authors on Leanpub. His first Leanpub book, D3 Tips and Tricks: Interactive Data Visualization in a Web Browser, has reached over 25,000 readers, and has grown to be over 600 pages in length.

You can follow Malcolm on Twitter @d3noob and read his blog at d3noob.org.

D3 Tips and Tricks

E: What is D3.js and why is it important?

M: The stock answer to that question could be that D3.js is a JavaScript library that allows the creation of web based graphics… and it does. But there are conservatively about a bazillion different ways of doing that, so there’s got to be something else that makes it popular. I thought I knew what the ‘special’ ingredient was for D3 but I’m pretty sure I was only partly right. I thought it was all about the ability for data to be abstracted from the presentation media. After all D3 is named for the phrase ‘Data Driven Documents’. If you provide a linkage between the data and the medium, when you change the data, the medium follows. It doesn’t really matter if it’s a static page that might change every time it gets refreshed or a visualization that has dynamic, interactive content. The data provides the driver for the document. Not the reverse. I have a friend who bemoans the World’s addiction to the A4 page (or perhaps the US Letter page for our American readers) and he’s absolutely right. We have traditionally formatted our documents to suit the notion of the printed page and in so doing we lose some of the ability for the content to drive the message. D3 made me consider that the data was the most important part of presenting information and D3.js is a library that provides enormous flexibility over what can be done with data. Here’s where the first bombshell for this interview gets dropped. It took me about a year and a half after starting to write D3 Tips and Tricks before I realised that I had missed arguably the most important aspect of D3. It’s ability to manipulate the DOM. I can’t really apologise for it since I had no real concept of it when I started writing. It wasn’t until I started to experiment with element, attribute and style control that I twigged that it’s not about graphics, it’s about all the content on the page. I know that will sound lame to those who understand such things, but I wrote D3 Tips and Tricks as a noob without any appreciation of the finer points of web technologies (all right, without any appreciation of some pretty fundamental aspects of web technologies). Perhaps that’s why I think that D3.js is important. Because it is an enabler in a lot of different ways, it has an ability to support a lot of different needs whether you’re a consumer of the end product or a producer.

E: How has the open nature of the D3 framework affected its evolution over time? What’s the community like?

M: In a lot of ways D3 has ‘shown the way’ for a range of different projects which are based on the same principles of representing data. D3 itself has evolved from the ‘Protovis’ project which was led by Mike Bostock and Jeff Heer of the Stanford Visualization Group (with significant help from Vadim Ogievetsky). Where D3 and Protovis have differed is in the type of visualizations they enable and the method of implementing them. While Protovis worked well with static representations, D3 added animation and interaction making complex and dynamic visualizations easier to implement. While the code is open for editing and contribution, there is little doubt that the driving force behind it is Mike Bostock. He has had fantastic support from Jason Davies who is an absolute genius at representing data (with a special penchant for geographical projections). But Mike is the guy. He drives the D3 bus, and like any situation where you have someone who is passionate and gifted at what they do, all of us in the D3 community are really comfortable with the direction he’s taking the project (IMHO). Starting to try any use D3 from scratch is hard. There’s a good reason that there’s so many tutorials and examples for people to follow. It’s not trivial to grok the way it works. At least that’s the way I found it. But at the same time I was really pleased at the time and patience that several of the leading lights in the D3 World had in providing advice. In particular I would single out Christophe Viau for being an all around good guy. Interestingly there is something of a ‘support group’ feel to the way that the community helps each other. That might be because the code can sometimes be ‘tricky’ or it might be because there is a ‘brothers/sisters in arms’ effect where we’re all fighting for a common cause. Certainly I have come across a range of people who were wanting to help with the production of D3 Tips and Tricks. Out of the blue I had contact from Filiep Spyckerelle and Robin Bennett who selflessly donated their time and expertise in proofreading and Jose ‘Tactician Jenro’ who crafted a fantastic cover for the book. I assume that they liked the idea that the book was available for free and they had talents that could help make it better for others.

E: What’s a particularly striking example of useful data visualization using D3? This is my favourite: http://www.brightpointinc.com/interactive/political_influence.

M: That’s a good example that highlights the way that D3 is often used to provide a data visualisation function. The exploration of data and the associated insights that D3 can support are some of the prime drivers for implementing and using the library. However, my favourite examples are mostly the ones that demonstrate something simple but cool. Pierre Guilleminot’s Fourier series visualisation combines a bunch of neat techniques to demonstrate a real world concept. Mike Bostock has a huge range of examples, but the simplicity and elegance of the point-along-path interpolation strikes a chord with me. I’m shamelessly going to include one of my own examples in here, because it was one of those ‘ah-ha’ moments when I learned something new and different about using D3 with HTML inputs and while it’s simple, it’s a good example of the continuous learning of cool stuff that seems to be the defacto standard when using D3. Lastly there’s a neat (recent) example from Elijah Meeks combining two different techniques to show a sankey diagram with particles to illustrate flow.

E: Who is the book meant for?

M: The book is actually aimed at me! I’ve formatted it so that the information is hopefully palatable for others, but it was originally written in a scruffy form so that I could remember the things that I was learning as I was experimenting with D3. Since I started off from scratch there was a lot to learn and so the information naturally steers towards newer users. The tragic thing is that because I don’t have any formal education in using these types of tools, I’m constantly afraid that I will be encouraging poor habits in others, but I figure that there’s enough really talented individuals in the community that can influence people in a positive direction so that any grievous errors I make along the way can be forgiven.

E: Are you still adding new content to the book?

M: In theory no, but in practice yes. That’s one of the neat things about Leanpub. Even though I think that I’ve done enough experimenting for the moment, I will often get comments pointing out places where I’ve messed up or things that can be improved. I regard the book as a living document. Kind of like a Wikipedia page that you can download and take away :-). I don’t have any firm plans to add more content, but if I find something new and exciting that I want to play with (and it looks useful) I’ll add it in.

Raspberry Pi: Measure, Record, Explore

E: For those who don’t know, what is Raspberry Pi and why is it important?

M: To paraphrase raspberrypi.org the Raspberry Pi is a low cost ($20 or $35 USD (there are a couple of versions)), credit-card sized computer that plugs into a computer monitor or TV, and uses a standard keyboard and mouse. In fact two days ago the Raspberry Pi foundation announced a new $5 version called the Pi Zero! It enables people to explore computing and programming and is capable of doing everything you’d expect a desktop computer to do, from browsing the internet and playing high-definition video, to making spreadsheets, word-processing, and playing games. But more interestingly, the Raspberry Pi has the ability to interact with the outside world via an array of different connectors and connection options. This makes the process of building a computer that can be used to ‘step outside itself’ really interesting and this is where things start to get cool. The low cost, open hardware and flexible connection options appears to have been a catalyst for the Raspberry Pi to become the defacto small board computer for educators, hackers (people who make stuff) and people who want to experiment with computers in a ‘low financial risk’ type way. There are a multitude of different types of projects that people have carried out to build stuff and learn about computers (http://www.instructables.com/id/Raspberry-Pi-Projects for a small sample).

E: Who is the book meant for?

M: Again it’s a book that I wrote mainly for myself because I wanted to learn more about the Raspberry Pi. However, because it wasn’t my first book I had a firm intention to try and make it easier for those who might also be interested in the topic. I’m not entirely sure that it’s worked out as well as I had wanted, but like any process, writing has been an exercise in learning more about trying to provide a better product to an audience. The book itself is aimed at those who are starting from scratch and who have ambitions of doing some fairly advanced things. I would hesitate to say that it can achieve that goal, but at the least it provides guidance and a degree of reassurance that even someone who has little to no knowledge (because that’s me) can still do something pretty cool. I could possibly be described as the poster child for perseverance narrowly defeating incompetence. In fairness there’s a bit of cross-over with D3 being used to display data captured by the Pi in the ‘Explore’ part.

E: Can you explain one of the projects that you set up in the book?

M: There are definitely some cool ones (IMHO) however, the one that gave me the most pleasure was definitely the first one that used a DS18B20 temperature probe to measure a single temperature. It wasn’t spectacular or difficult or unique. But it was the first time I had tried to do something like connecting an external sensor to a computer so it was a formative moment that showed me that the process wasn’t rocket science. Better than that, it was achievable to pretty much anyone. The big key for me was that once I had achieved this first goal, I knew that getting more complicated projects was going to be achievable as well. For anyone interested in playing with a Raspberry Pi I cannot stress enough that the key moment will come after that first ‘win’. Once you have passed through that barrier the possibilities are endless. Using a temperature probe like a DS18B20 is pretty simple, but there are others that would fit the bill as well.

E: In your book there’s a cool example of web scraping that involves a couple of Leanpub books. Can you talk a little bit about what web scraping is, and how it can be useful?

M: Yeah, good question. Web scraping is a practice that involves programmatically getting a computer to ‘browse’ to a web page and then to extract some form of data from it. I had never done it before, and I had a strong desire to see how the sales of books on Leanpub compared to each other. Leanpub has a great API for authors to check the data about their own books, but if you want to get a historical trend of sales over time for another author’s books it can’t be done simply. However, because quite a number of authors display the number of sales on the main pages of their books, this represents a piece of data that varies as the book sells more copies. So in this case I have a small script running at a regular interval on a Raspberry Pi that ‘browses’ to the web page, reads the specific piece of the page that has the number of sales and then records the information in a database. The script does this for a few books of note and it does it every hour. Over time this has the effect of building up a good historical record of sales. So here’s the cautionary tale…. Web scraping is a double-edged sword that need to be used carefully. On one hand, used responsibly, it can select data at a reasonable interval without upsetting anyone. On the other hand, if someone scrapes a page constantly, this is a major encumbrance to the web site owner who is having to pay for bandwidth that is created. Additionally the activity may be blocking genuine visitors from getting access. It is also widely accepted that scraping a website to plagiarise a company or person’s data for some evil purpose like monetary gain or theft of intellectual property is a pretty reasonable grounds for being sued, so for those who might be considering doing this, make sure your purposes are for good and not evil!

Working with Leanpub

E: In both your D3.js book and your Raspberry Pi book you write about using a story-like approach to your explanations, rather than a “do this” approach. Can you explain why you made this choice? What kind of feedback around this have you had from readers?

M: I should probably expand on the ‘story-like’ description a bit. Any project is essentially a journey where you have starting position, a series of activities and an end point. In the process of trying to learn about how to do things with D3 or the Raspberry Pi I was constantly looking for examples or information to help move the project forward. In a lot of cases the information I was reading had a fairly high level of assumed knowledge. This meant that where something on the web simply said ‘do this’, I often found myself needing to learn what the hell was going on so that I could move my own journey forward. Since I was learning something, it made sense to write it down and I wanted to do this in a way that felt comfortable. My writing style is to try and make the tone conversational and light hearted. There were many occasions where I was carrying out an activity that I had to describe as ‘magic’ and as a result I have had several instances where readers have contacted me and explained the error of my ways or offered excellent advice on how to understand things using a simpler method. This is really just another way of saying that the feedback that I’ve received from the book has been almost universally positive. I have a bit of a theory about why this is, but sadly it has nothing to do with my outstanding writing ability or technical mastery. I’m pretty sure it’s because I give the books away. Since people can essentially pay what they want for the books (including nothing) I’d like to think that they’re getting value for money :-). And it’s pretty easy to be positive about something that you got for the right price.

E: Your D3.js Tips and Tricks has over 25,000 readers. Could you explain how you managed to reach so many people?

M: It’s pretty crazy really. I’m just some random guy that wrote some information while experimenting with some computer code, so to have that downloaded 25,000 times is a bit surreal. I think that there’s some good reasons why its been popular and there’s some bad reasons as well. One of the good reasons is that when I started writing about D3 there were only a couple of books out on it (Mike Dewar (Getting Started with D3) and Scott Murray (Interactive Data Visualization for the Web)) and while they are excellent books (they REALLY know their stuff), the number of choices were small. The other good reason was the linkage from d3noob.org where I was posting information in a blog while I was writing the book. As you might imagine there was quite a bit of information on the blog and I noted the references to Leanpub on pretty much each post. That definitely helped point people in the right direction and it works out really well for readers since the Leanpub version of the information is always going to be the most up to date. The bad reason why it’s popular is the price. It’s hard to say no to something free! So while it’s impressive to say that 25,000 people have downloaded the book, they may well have done so simply because there was no financial risk in doing it. Whether it was read or even of any use is another matter:-).

E: Is engaging directly with people who have already bought your book important to you, and is there more we could do at Leanpub to help you engage with your readers online?

M: I know that it should be important to me, but like anyone with time pressures, more often than not I find myself with little to no opportunity to be able to respond in a timely fashion to questions. It’s really neat to be able to help someone with a problem, but in a lot of cases it’s a case of the blind leading the blind, so I often advise people to seek assistance on Stack Overflow in the first instance. There are some really sharp folks there willing to help out. I’m not really sure what more Leanpub could do to improve engagement. You already provide an extraordinary service that helps authors reach out in a range of media types and ways. Forums and blogs stand out as the default responses, but you would only be reinventing a wheel that may only complicate the experience. It’s a really good question and I’d like to think that authors and readers alike could chime in here at any time.

E: How did you like Leanpub’s workflow?

M: I think that the secret of success for Leanpub’s workflow is twofold. Firstly there is no single way that an author has to work to create a book. The fact that there are a range of different options, all the way from adding content in a browser to using Dropbox and GitHub means that the barrier for entry is reduced for those who need it and there is greater flexibility and utility for those who want more control. I mainly use Dropbox and find the experience pretty easy. To be perfectly honest, the method just ‘gets out of the way’ and a writer can focus on more important things like adding content. It’s a measure of the ease of use that my main concern is the editor that I use. Markdown totally rocks, and I’m a big fan of MarkdownPad for writing. But it’s only available on Windows. When travelling I use Ubuntu on my laptop and I’ve yet to find a Markdown editor that comes close to MarkdownPad in Linux. I’ve used the ‘in browser’ editor when helping put together the short book ’A history of Pye Radio Ltd. in Waihi, New Zealand’ and I while I didn’t find the experience as easy, it was awesome to have the flexibility to work from any OS that supported a browser. Ultimately however, the true test of ease of use lies in the fact that Leanpub makes the process of authoring and publishing easy. I am constantly astonished that more people don’t take the opportunity to put information they have squirrelled away into a book. The iterative process that Leanpub allows means that you can afford to get information pushed out quickly and then tune it or get feedback to improve it’s value. For me a big part of this is that Leanpub allows setting a price that starts at $0. When you think about it, that’s a big deal for a company. There’s a lot of support that they have to provide with little to no promise of a return. Part of my reasoning for writing is to try and give something back to the open source communities that have supported the interesting stuff that I enjoy, so to be able to distribute my books for free is just flat out perfect. I’m really stoked to see that the process is paying off for them as well (pun intended). While people have been kind enough to donate enough to buy me the occasional beer to celebrate, it’s a hobby and not going to pay the bills any time soon. However, folks like Roger Peng, Jeff Leek and Brian Caffo have found a niche for distributing books to support their courses which allow their students to get the work for free and in the process, the sheer scale and generosity of the people downloading the books sees them fairly high on the Leanpub earnings pages. That rocks!

E: Leanpub was started by authors who were really opinionated about their writing and publishing tools. How can we improve these, from your perspective?

M: I’m struggling to find a good answer to this. I’m just so grateful that Leanpub runs the service I would be more than happy to write in the dirt with a stick. I know that people will have a preference for doing work in a particular way, but I’m happy to be flexible and the Markdown format provides a pretty vanilla mechanism to allow variation in the type of editor used. If someone’s interested in writing their books in Microsoft Word, that’s fine, but I would be surprised if in 10 years or so we found the same sort of reliance on proprietary document formats that we do today. Life is just so much easier in plain text. Another point to add is the great work that is going on to support Markua. That has the potential to provide a great base for easier distribution and editing of information.

E: Is there anything else about Leanpub that you think we could improve?

M: I’m a Leanpub fanboi, but there are a few of things that spring to mind that I know have been explored before and there are good reasons for and against, but they’re things that would make a difference to me. I’d also like to preface my thoughts with the observation that Leanpub is always super active in dealing with online discussions and being engaged with their author community. Kudos.

Firstly, some sort of ‘hold my hand’ path for production of a hard copy version of a book. I know that Leanpub makes print ready PDF versions, and they acknowledge that the concept of a hard copy service detracts from their core business. I have read recommendations for options to take Leanpub books through different services, but I’d be seriously interested if one partnered with Leanpub to guide the process. It’s really just me being apprehensive about the process. I’ve trusted Leanpub to make my words look pretty in a digital format and they do a great job. I don’t have that same faith for taking the step to print.

Secondly, an option to produce an index for a book. I know the concept isn’t trivial and I’ve read the online discussions and there are good arguments in both directions. But the ability to add an index would be useful IMHO.

Thirdly I’d really like to get access to extra data on other books. I don’t think that there’s really any motivation for Leanpub to do it, but an API that allowed me to view sales numbers of different books, page counts, percentage completion, price, that sort of thing, à la the process that I engineered by scraping would be neat. I’m not going to hold you to it since that would be a bunch of work for you guys with little to no reward :-).

Fourth: (this is starting to add up) I’d like to be able to add or remove books from a collection without having to essentially recreate the collection from scratch. There must be some catch that makes this tricky (I assume that it’s the approvals from different authors with percentages and suchlike).

E: My last question is, are you working on any more books?

M: Well yes. I have a book on Leaflet that is really only about 50% finished, and has been so for quite a while. I’d like to think that I will revisit it when they publish version 1.0 (which should be soon). I’ve also been helping out with a concept of publishing a series of books aimed at providing enough information on a topic to help people get going. I have no idea where it will head, but at this stage the ‘Just Enough’ series includes Just Enough Linux, Just Enough Co-Authoring in Leanpub, Just Enough Raspberry Pi, Just Enough ownCloud on a Raspberry Pi and Just Enough Ghost on a Raspberry Pi. I’d like to add more to this lineup and the idea is to bundle them so that folks can just download the whole kit and caboodle in one hit. At the moment we have them in the ’Just Enough Collection’ for the staggering price of $0.99.

Parting thoughts

M: Thanks for reaching out to ask if I was interested in participating in the interview. It’s been an opportunity for a bit of reflection that has been very therapeutic. I’ve found that I enjoy the process of writing. Leanpub has provided me with an outlet to do something that I never imagined I would be enthusiastic about and in the process I hope that others have gotten some benefit as well.

Recent Posts

Leanpub Podcast Interview #23: Patrick Kua

Patrick Kua

Leanpub's Year In Review: Looking Back on 2015 And Looking Forward To 2016

The Top Books On Leanpub In 2015

New Leanpub Features and Initiatives In 2015

Packages

Leanpub Podcast Reboot

Shopping Cart

Our Brief Experiment With An Affiliate Program

Buying And Distributing Multiple Copies of Ebooks on Leanpub

Bring Your Own Book

A New Role for Len at Leanpub

Looking Ahead To 2016

Thanks!

The Top Ten Most Popular Leanpub Books In 2015

Leanpub’s 2015 Top Ten Most Popular Books List

#10. Data Analysis for the Life Sciences by Rafael A Irizarry and Michael I Love

#9. PHP: The “Right” Way: Your guide to PHP best practices, coding standards, and authoritative tutorials by Phil Sturgeon and Josh Lockhart

8. Report Writing for Data Science in R by Roger D. Peng

7. D3 Tips and Tricks: Interactive Data Visualization in a Web Browser by Malcolm Maclean

6. Regression Models for Data Science in R: A companion book for the Coursera Regression Models class by Brian Caffo

5. Statistical inference for data science: A companion to the Coursera Statistical Inference Course by Brian Caffo

4. Exploratory Data Analysis with R by Roger D. Peng

#3. The Art of Data Science: A Guide for Anyone Who Works with Data by Roger D. Peng and Elizabeth Matsui

#2. The Elements of Data Analytic Style: A guide for people who want to analyze data by Jeff Leek

1. R Programming for Data Science by Roger D. Peng

The Top Ten Bestselling Leanpub Books In 2015

Leanpub’s 2015 Top Ten Bestseller List

#10. Ansible for DevOps: Server and configuration management for humans by Jeff Geerling

#9. Exploring ES6: Upgrade to the next version of JavaScript by Axel Rauschmayer

#8. The Art of Data Science: A Guide for Anyone Who Works with Data by Roger D. Peng and Elizabeth Matsui

#7. Software Architecture for Developers: Technical leadership by coding, coaching, collaboration, architecture sketching and just enough up front design by Simon Brown

#6. Exploratory Data Analysis with R by Roger D. Peng

#5. Build APIs You Won’t Hate: Everyone and their dog wants an API, so you should probably learn how to build them by Phil Sturgeon

#4. ember 101: Learn Ember.js with ember-cli by Adolfo Builes

#3. MEAN Machine: A beginner’s practical guide to the JavaScript stack by Chris Sevilleja and Holly Lloyd

#2. The Elements of Data Analytic Style: A guide for people who want to analyze data by Jeff Leek

1. R Programming for Data Science by Roger D. Peng

A Leanpub Interview With Malcolm Maclean

D3 Tips and Tricks

Raspberry Pi: Measure, Record, Explore

Working with Leanpub

Posts