Oct 06, 2010
Chris Wanstrath, Github
Collaborative Softwrae Development in teh 21st century
We do collaborative software development. This is the way software is usually developed. But a lot of the methods were developed in the 1990s.
Linus Torvalds. Linux. Which is everywhere now. Firefox, another really successful open source project. Google Chrome, which is another, based on the Webkit project, from Apple. Which also powers Safari, the iTines store, as well as a whole bunch of smart phones.
Open sources is a success, we all know that, it's nothing new.
But how does open source software development work? How do you organize that, which evolved from chaos, with no CEO.
(Flow diagram of how to create a Firefox patch)
Preparation -> Module ownership -> Creating a Patch - >Testing -> Gettkng Reviews -> Committing the Patch -> You are now immortal
The two parts where there's no code are 'module ownership' and 'peer review'.
Linux - a hierarchical structure. Linus at the top. A dozen or so lieutenants. And then the great unwashed. This all evolved out of the needs of the community. So Linus delegated to trusted associates. But Linus is still a developer.
Firefox - modules have owners. Really, what this means is, modules have editors' - I oversee all the content, I make sure it's the right format, etc. - code review is like peer review. Almost every open source project uses peer review.
A patch, to me, is a dead butterfly - it's just a slice of time. What's interesting is living butterflies. What I mean is taking the social networks like Facebook and Twitter.
Diagram - 'coverall' the lines represent development. Each line is a separate branch of development. We go to this other world of revision control and fancy graphics to see this living butterfly. What we're doing with github is taking this to the next level to get more butterflies.
What Wikipedia is that model applied to an encyclopedia. What github is it applied to code. I can see the changes made over time, just like Wikipedia. See the changes, the actual diff. Start a discussion - an ad hoc Wikipedia.
Now - what if we had a hundred butterflies. What github does is allow your entire open source development is open? Then you push your changes to that copy, which everyone can see, and then when you;re ready, create a patch, or if you want, just pass a link. Basically, if I want to work on Firefox, I fork it. Then I just work on it. And then anyone can look, comment, add, etc.
(What about overwiring code)
This is called distributed version control. In the classical model there is a canonical version. In our system there are multiple versions. Nothing has changed but the visibility. Then eventually, we just merge them back in. Yes, there's a huge aspect of human curation, but the system makes it easier. Example of Iceweasel, which is the Debian fork.
(You have a much more flexible process, at the human level, just because of the visibility).
(I can see fear on the faces of managers)
Do we see this in enterprises? yes. Linux is a big serious project with a lot of money. What we're interested in at github is where we take all these ideas, and put them inside a firewall, with permissions, etc., but, for other things, why not share them inside a company. Cut down on duplication of efforts. What we put on top of all this is access control.
Forks enable collaboration. There aren't those formal rigid processes that keep you from innovating. Maybe you just want to try something out. But with github there's a place to do that, try it out, and discuss whether it belongs in the main code.
Some examples:
node.js - 3311 watchers, 345 forks. They care about every line of code in node.js - they;re very technical people.
jQuery - 4454 watchers, 390 forks. Started out in the formal model, now a very dynamic open source project.
homebrew - 2818 watchers, 1123 forks. All about contributing recipes that do what you want them to do.
One of the great things about this process is attribution. (Great video of the Ruby on Rails collaboration network after it joined github).
Yesterdaym 900 projects were forked. 50,000 changes pushed. This is boxes of butterflies.
We think that people prefer live butterflies. It's a better way to work. It started with open source. It's a better way to work. It gets rid of ownership, inflighting.
(Challenges of open source in interprise?) besides lawyers and IP?
Big project - open source with RIM. But the open source methodology is clay. You can shape it how you want to. Open source survives influx - but may be not all do. The biggest thing we fight is the mindset that everything we do is perfecvt, because we know it isn't.
What about private projects? Well, what about them? I can't insert code into Linux. I can't insert into your personal code. All you do is take the visibility, and flip it from public to private. We even have some people who own projects that have their own fork of the project, because they don't want to experiment with the canonical.
Open source, and peer review, create some really interesting and useful things.
(It's not just visibility, though, right?)
Yes - there's a lot of high tech going on here. Stuff even from Linus. But it's when I saw the visible that it all came together for me. The technology is all based on the visualization of version control.
We don't care about facebook and how people are optimizing their relationships with their friends. We care about the power things like Facebook give you.
David Fetherstonhaugh, IDEO
Reputation is the New Black... Box
This is a nod to the new algorithmic reputation systems now coming online today. Riley - temporary links are the innovation that will happen. But a person asked, "where's the business modelhere". My presentation addresses both things.
12 years ago, Harvard graduate Mark Bening (sp?) had a great idea. He though, if eBay can create a market for stuff, we can create a market for non-stuff - expertise, etc. He launches exp.com - exp for expert. But it turns out there are really tough problems to solve.
The second problem - the transaction platfrorm - is the linchpin that makes it work.
Google, exp.com and Gerson Lehrman Group (an offline expertise hiring system)
They're all basically decision engines. They reduce uncertainty by gaining information you didn't have beforehand. Each serves a different spectrum of the market.
Google - very good, fast, smart - but not smarter than a fifth grader. The Google inbox guesses at what you want.
Gerson Lehrman is pretty fast - a couple of days - and really smart. But it's not scalable.
exp.com is all of these. But it's a failed company. It owes venture capitalists millions -VCs have thrown millions at this space. It's odd - we can have anyone online meet anyone online almost instantly, basically for free. But to have them enter a transaction - it puts all sorts of risks on the buyers, plus all the resources to vet people, only to find you've hired a false positive.
We're missing the big middle market.
Why do these things fail? Why do knowledge networks fail? Reference to George Akerlof, who wrote on the lemon problem - buyers who enter a market, who are willing to buy, but who fear lemons, and sellers of good products, but can't sell because buyers are afraid, and all that's left are lemon sellers and the suckers that buy them.
We don't understand trust very well, or how to design reputation systems that solve this problem. Some examples to show how they fail...
One way: reputation is social collateral. Your reputation is your 'security' (for a loan, transaction, etc.). Reputation is a form of social control. Suppose Hal burst into song in the middle of a conference - he won't do that, because he might be lambasted and kicked out, so he won't do that. You are holding this collateral. But... Chatroulette. This is a site without a reputation system. Tere's no collateral given - people are free to act and there's nothing to lose. Or, Yelp it has a five star system to find restaurants, etc. But we pay more attention to the comments. This is a demonstration of the failure of the reputation system. It works OK for restaurants, but not for $100K decisions. I might add that Google had a Google Answers system, and they closed it down, because it wsn't scalable.
The NY Stock Exchange - when you buy a stock, this is a reputation system. It is based on a reputation created by market processes. We don't wonder whether people are gaming the system. We agree that these are representative of the company's value. We want our reputation system to be like that.
(So - why did facebook work?)
Well - like eBay or Google search, is a winner-take-all market that may be independent of quality. Diaspora may be better than Facebook, but they may not succeed because of this.
So - knowledge exchanges will feel just like stock markets.
Three design principles that led to our solution:
1. Couple reputation with behaviour. Higher ranks, the more you can do. Military ranks, frequent flyer plans. Different levels of reputation give you access to different levels of resources. Credit score, for example - a higher credit score gives you access to more credit. or, from World of Warcraft, in-game privileges. The motivation occurs intrinsically, without cognition. That's how it should work in a reputation system.
2. Incentivize honest peer review. All the peer review processes suffer from the same thing - the freeloader problem. We want to enjoy ratings, but we won't spend our time rating them. So the ratings are not very good because of this dynamic. The goal is to incent people to rate other reviewers. So yo wouldn't collude because your reputation would suffer.
3. Think beyond identity. We tend to think of reputation through identity, so we tend to think that identity is central to reputation, but a robust reputation in knowledge exchanges do not depend on identity. Eg., again, gaming experience. It's not about who you are, it's about what you do. That's what you care abut as a buyer of expertise. Reputation through identity is a rabbit hole. "Are you the person you say you are." And you get the whole system of Federated ID. Then, the question is, do you have the expertise. But - as a buyer, I still don't know if you will do what I need.
The reputation statistic is essentially a summary of all your previous transactions in a market. A lot like the value of a stock in the stock market. It isn't who you are. It's what you do.
We tend to think of knowlddge networks as a part of social networks. LinkedIn, for example - we thionk of it that way. But it has a failed expertise system, 'LinkedIn Answers'. Rather, we're after 'reputation through algorithms'.
the inputs to the black box: expert behaviour, buyer rating, peer review.
Example: stackoverflow. There's lots wrong with this, not robust, but it's an interesting application of some of the principles.
Why does this matter? At IDEO we are approached by lots of firms that want us to design exchanges. There's lots of other companies - like Intuit. They for years have been trying to figure out how to tap the experts to support small businesses.
This is probably a much larger market even than eBay, but what's really interesting are the social implications. It may result in a fairly massive redistribution of wealth from developed countries to developing countries. There are very qualified people anxious and willing to work at a quarter of the price.
(Reputatoon is like brand? Is it something you establish on your own, a reflection of how other people see you? But it's also evident that people work really hard to build a reputation - that's were it becomes confusing. To what degree can reputation become self-driven?) Brand is really about identity. In a pure knowledge network, it's not really about brand, it's about cold hard evaluation of your performance. It would operate best anonymously.
Bob Lucky, TTI Vanguard
Conference Reflections
We heard about reputation and social capital and you;'re looking at someone who has lost it all - a bunch of people got one of those emails from me "I'm stuck in London, please wire me some money." I fell for a scam, lost access to my email, and had the email sent out. Then, someone else wrote me, "You don't know me, the hacker is in Nigeria, you're in Vancouver, and here is your password." Heh.
Warning - don't fall from the "confirm a purchase from iTunes" email. I'm afraid to click on anything these days.
Some things I took away.
John Kelly's talk - the pictures of the networks. What is the real meaning here. We see isolated communities and bridges - the US looks so homogenized. I'm not sure we're looking at the right thing.
I'm fascinated with the office stuff. From ownership of individual offices to shared space. Proprietary business models to peer production.
Ted Rybeck - the things is it's so surprising that so many people are interested in Oreo cookies.
Susan Bonds - crafting thes devious games. Who has all this time?
The museum director - who deals in tangibleheritage - who had no slides for his talk - the relation between tangible and intangible cultural heritages.
Lili Cheng - a lot of stuff going on. Making email better. It would be better if it were better. Do you really trust Outlook to surface important email?
Brian Behelendorf, with open government. I have trouble with open government. But I had experience - every meeting had to be open. I thought it was great. But all you get are special interests and crazies. And you saw this in this talk because of the birther thing.
Erin McKean - interesting to see someone who cares about words. Really cares about words. I thought about looking up some words - but then thought - I'll never see this word again.
Riley Crane - I was pretty interested in this. But the real answer was in the end - "we were really lucky." What was interesting was putting together a social network, Crafting viral messages. they got picked up by Slashdot - but it turns out they did that themselves.
Craig Newmark - a highlight. Maybe he was lucky too - it's a winner-take-all thing, and mayb e he's just the one who won. I was really charmed by his humility. And he said, "I used to do things, now I just talk about them."
Fernando Flores - the same charm as Craig had. A sense of purpose, commitment. Other people seem to be faking it, but you are really doing it. How to deal with trust.
Me - picked up from more technical talks in London about how networks work. Principles of association. Cascade phenomena. Network death - loss of plasticity in the network.
Jordan Raddick. Looking at the galaxies. It's amazing that people want to do these kind of things. But people want to contribute to the scientific process. It's like SETI @ Home. Could you crowdsource looking at Afghanistan?
Donna Cuomo - handshake from MITRE. Not many companies with something similar to this? Will people spend more time screwing around with this than working. We don't have metrics, we can't tell whether it's helping.
John Perry Barlow - a really uplifting talk. Heh.
Github - Chris Wanstrath - this might be a takehome, to see if you can use this for software development in your company.
David Fetherstonhaugh - would I want to change last names with him? But a great talk! It cogently and concisely addressed one of the key issues of the meeting.
Collaborative Softwrae Development in teh 21st century
We do collaborative software development. This is the way software is usually developed. But a lot of the methods were developed in the 1990s.
Linus Torvalds. Linux. Which is everywhere now. Firefox, another really successful open source project. Google Chrome, which is another, based on the Webkit project, from Apple. Which also powers Safari, the iTines store, as well as a whole bunch of smart phones.
Open sources is a success, we all know that, it's nothing new.
But how does open source software development work? How do you organize that, which evolved from chaos, with no CEO.
(Flow diagram of how to create a Firefox patch)
Preparation -> Module ownership -> Creating a Patch - >Testing -> Gettkng Reviews -> Committing the Patch -> You are now immortal
The two parts where there's no code are 'module ownership' and 'peer review'.
Linux - a hierarchical structure. Linus at the top. A dozen or so lieutenants. And then the great unwashed. This all evolved out of the needs of the community. So Linus delegated to trusted associates. But Linus is still a developer.
Firefox - modules have owners. Really, what this means is, modules have editors' - I oversee all the content, I make sure it's the right format, etc. - code review is like peer review. Almost every open source project uses peer review.
A patch, to me, is a dead butterfly - it's just a slice of time. What's interesting is living butterflies. What I mean is taking the social networks like Facebook and Twitter.
Diagram - 'coverall' the lines represent development. Each line is a separate branch of development. We go to this other world of revision control and fancy graphics to see this living butterfly. What we're doing with github is taking this to the next level to get more butterflies.
What Wikipedia is that model applied to an encyclopedia. What github is it applied to code. I can see the changes made over time, just like Wikipedia. See the changes, the actual diff. Start a discussion - an ad hoc Wikipedia.
Now - what if we had a hundred butterflies. What github does is allow your entire open source development is open? Then you push your changes to that copy, which everyone can see, and then when you;re ready, create a patch, or if you want, just pass a link. Basically, if I want to work on Firefox, I fork it. Then I just work on it. And then anyone can look, comment, add, etc.
(What about overwiring code)
This is called distributed version control. In the classical model there is a canonical version. In our system there are multiple versions. Nothing has changed but the visibility. Then eventually, we just merge them back in. Yes, there's a huge aspect of human curation, but the system makes it easier. Example of Iceweasel, which is the Debian fork.
(You have a much more flexible process, at the human level, just because of the visibility).
(I can see fear on the faces of managers)
Do we see this in enterprises? yes. Linux is a big serious project with a lot of money. What we're interested in at github is where we take all these ideas, and put them inside a firewall, with permissions, etc., but, for other things, why not share them inside a company. Cut down on duplication of efforts. What we put on top of all this is access control.
Forks enable collaboration. There aren't those formal rigid processes that keep you from innovating. Maybe you just want to try something out. But with github there's a place to do that, try it out, and discuss whether it belongs in the main code.
Some examples:
node.js - 3311 watchers, 345 forks. They care about every line of code in node.js - they;re very technical people.
jQuery - 4454 watchers, 390 forks. Started out in the formal model, now a very dynamic open source project.
homebrew - 2818 watchers, 1123 forks. All about contributing recipes that do what you want them to do.
One of the great things about this process is attribution. (Great video of the Ruby on Rails collaboration network after it joined github).
Yesterdaym 900 projects were forked. 50,000 changes pushed. This is boxes of butterflies.
We think that people prefer live butterflies. It's a better way to work. It started with open source. It's a better way to work. It gets rid of ownership, inflighting.
(Challenges of open source in interprise?) besides lawyers and IP?
Big project - open source with RIM. But the open source methodology is clay. You can shape it how you want to. Open source survives influx - but may be not all do. The biggest thing we fight is the mindset that everything we do is perfecvt, because we know it isn't.
What about private projects? Well, what about them? I can't insert code into Linux. I can't insert into your personal code. All you do is take the visibility, and flip it from public to private. We even have some people who own projects that have their own fork of the project, because they don't want to experiment with the canonical.
Open source, and peer review, create some really interesting and useful things.
(It's not just visibility, though, right?)
Yes - there's a lot of high tech going on here. Stuff even from Linus. But it's when I saw the visible that it all came together for me. The technology is all based on the visualization of version control.
We don't care about facebook and how people are optimizing their relationships with their friends. We care about the power things like Facebook give you.
David Fetherstonhaugh, IDEO
Reputation is the New Black... Box
This is a nod to the new algorithmic reputation systems now coming online today. Riley - temporary links are the innovation that will happen. But a person asked, "where's the business modelhere". My presentation addresses both things.
12 years ago, Harvard graduate Mark Bening (sp?) had a great idea. He though, if eBay can create a market for stuff, we can create a market for non-stuff - expertise, etc. He launches exp.com - exp for expert. But it turns out there are really tough problems to solve.
The second problem - the transaction platfrorm - is the linchpin that makes it work.
Google, exp.com and Gerson Lehrman Group (an offline expertise hiring system)
They're all basically decision engines. They reduce uncertainty by gaining information you didn't have beforehand. Each serves a different spectrum of the market.
Google - very good, fast, smart - but not smarter than a fifth grader. The Google inbox guesses at what you want.
Gerson Lehrman is pretty fast - a couple of days - and really smart. But it's not scalable.
exp.com is all of these. But it's a failed company. It owes venture capitalists millions -VCs have thrown millions at this space. It's odd - we can have anyone online meet anyone online almost instantly, basically for free. But to have them enter a transaction - it puts all sorts of risks on the buyers, plus all the resources to vet people, only to find you've hired a false positive.
We're missing the big middle market.
Why do these things fail? Why do knowledge networks fail? Reference to George Akerlof, who wrote on the lemon problem - buyers who enter a market, who are willing to buy, but who fear lemons, and sellers of good products, but can't sell because buyers are afraid, and all that's left are lemon sellers and the suckers that buy them.
We don't understand trust very well, or how to design reputation systems that solve this problem. Some examples to show how they fail...
One way: reputation is social collateral. Your reputation is your 'security' (for a loan, transaction, etc.). Reputation is a form of social control. Suppose Hal burst into song in the middle of a conference - he won't do that, because he might be lambasted and kicked out, so he won't do that. You are holding this collateral. But... Chatroulette. This is a site without a reputation system. Tere's no collateral given - people are free to act and there's nothing to lose. Or, Yelp it has a five star system to find restaurants, etc. But we pay more attention to the comments. This is a demonstration of the failure of the reputation system. It works OK for restaurants, but not for $100K decisions. I might add that Google had a Google Answers system, and they closed it down, because it wsn't scalable.
The NY Stock Exchange - when you buy a stock, this is a reputation system. It is based on a reputation created by market processes. We don't wonder whether people are gaming the system. We agree that these are representative of the company's value. We want our reputation system to be like that.
(So - why did facebook work?)
Well - like eBay or Google search, is a winner-take-all market that may be independent of quality. Diaspora may be better than Facebook, but they may not succeed because of this.
So - knowledge exchanges will feel just like stock markets.
Three design principles that led to our solution:
1. Couple reputation with behaviour. Higher ranks, the more you can do. Military ranks, frequent flyer plans. Different levels of reputation give you access to different levels of resources. Credit score, for example - a higher credit score gives you access to more credit. or, from World of Warcraft, in-game privileges. The motivation occurs intrinsically, without cognition. That's how it should work in a reputation system.
2. Incentivize honest peer review. All the peer review processes suffer from the same thing - the freeloader problem. We want to enjoy ratings, but we won't spend our time rating them. So the ratings are not very good because of this dynamic. The goal is to incent people to rate other reviewers. So yo wouldn't collude because your reputation would suffer.
3. Think beyond identity. We tend to think of reputation through identity, so we tend to think that identity is central to reputation, but a robust reputation in knowledge exchanges do not depend on identity. Eg., again, gaming experience. It's not about who you are, it's about what you do. That's what you care abut as a buyer of expertise. Reputation through identity is a rabbit hole. "Are you the person you say you are." And you get the whole system of Federated ID. Then, the question is, do you have the expertise. But - as a buyer, I still don't know if you will do what I need.
The reputation statistic is essentially a summary of all your previous transactions in a market. A lot like the value of a stock in the stock market. It isn't who you are. It's what you do.
We tend to think of knowlddge networks as a part of social networks. LinkedIn, for example - we thionk of it that way. But it has a failed expertise system, 'LinkedIn Answers'. Rather, we're after 'reputation through algorithms'.
the inputs to the black box: expert behaviour, buyer rating, peer review.
Example: stackoverflow. There's lots wrong with this, not robust, but it's an interesting application of some of the principles.
Why does this matter? At IDEO we are approached by lots of firms that want us to design exchanges. There's lots of other companies - like Intuit. They for years have been trying to figure out how to tap the experts to support small businesses.
This is probably a much larger market even than eBay, but what's really interesting are the social implications. It may result in a fairly massive redistribution of wealth from developed countries to developing countries. There are very qualified people anxious and willing to work at a quarter of the price.
(Reputatoon is like brand? Is it something you establish on your own, a reflection of how other people see you? But it's also evident that people work really hard to build a reputation - that's were it becomes confusing. To what degree can reputation become self-driven?) Brand is really about identity. In a pure knowledge network, it's not really about brand, it's about cold hard evaluation of your performance. It would operate best anonymously.
Bob Lucky, TTI Vanguard
Conference Reflections
We heard about reputation and social capital and you;'re looking at someone who has lost it all - a bunch of people got one of those emails from me "I'm stuck in London, please wire me some money." I fell for a scam, lost access to my email, and had the email sent out. Then, someone else wrote me, "You don't know me, the hacker is in Nigeria, you're in Vancouver, and here is your password." Heh.
Warning - don't fall from the "confirm a purchase from iTunes" email. I'm afraid to click on anything these days.
Some things I took away.
John Kelly's talk - the pictures of the networks. What is the real meaning here. We see isolated communities and bridges - the US looks so homogenized. I'm not sure we're looking at the right thing.
I'm fascinated with the office stuff. From ownership of individual offices to shared space. Proprietary business models to peer production.
Ted Rybeck - the things is it's so surprising that so many people are interested in Oreo cookies.
Susan Bonds - crafting thes devious games. Who has all this time?
The museum director - who deals in tangibleheritage - who had no slides for his talk - the relation between tangible and intangible cultural heritages.
Lili Cheng - a lot of stuff going on. Making email better. It would be better if it were better. Do you really trust Outlook to surface important email?
Brian Behelendorf, with open government. I have trouble with open government. But I had experience - every meeting had to be open. I thought it was great. But all you get are special interests and crazies. And you saw this in this talk because of the birther thing.
Erin McKean - interesting to see someone who cares about words. Really cares about words. I thought about looking up some words - but then thought - I'll never see this word again.
Riley Crane - I was pretty interested in this. But the real answer was in the end - "we were really lucky." What was interesting was putting together a social network, Crafting viral messages. they got picked up by Slashdot - but it turns out they did that themselves.
Craig Newmark - a highlight. Maybe he was lucky too - it's a winner-take-all thing, and mayb e he's just the one who won. I was really charmed by his humility. And he said, "I used to do things, now I just talk about them."
Fernando Flores - the same charm as Craig had. A sense of purpose, commitment. Other people seem to be faking it, but you are really doing it. How to deal with trust.
Me - picked up from more technical talks in London about how networks work. Principles of association. Cascade phenomena. Network death - loss of plasticity in the network.
Jordan Raddick. Looking at the galaxies. It's amazing that people want to do these kind of things. But people want to contribute to the scientific process. It's like SETI @ Home. Could you crowdsource looking at Afghanistan?
Donna Cuomo - handshake from MITRE. Not many companies with something similar to this? Will people spend more time screwing around with this than working. We don't have metrics, we can't tell whether it's helping.
John Perry Barlow - a really uplifting talk. Heh.
Github - Chris Wanstrath - this might be a takehome, to see if you can use this for software development in your company.
David Fetherstonhaugh - would I want to change last names with him? But a great talk! It cogently and concisely addressed one of the key issues of the meeting.
