Tuesday, 12 November 2013

The Means of Production are Decentralizing: the MAID and the MOPD clean up the MOPPPO


This is a commentary on the Implications of the Internet for the SocioEconomic Configuration of the Future (IISECF). It proposes that the old question of the MOPPPO, namely, of whether or not the Means Of Production should be Publicly or Privately Owned, is much less relevant today, because the MAID (Means of Acquiring Information have Decentralized) and the MOPD (Means of Production are Decentralizing) are cleaning it up.

The MAID and the MOPD clean up the MOPPPO

Classic socioeconomic debate often assembles itself about a the question of whether or not the Means Of Production should be Publicly or Privately Owned (MOPPPO). Most attempts to answer this question quickly dissolve into a seeping sand storm of confusion about the role of an entity notoriously known as, "The Government." What most discussions don't seem to take seriously in the debate is the Implication of the Internet in the SocioEconomic Configuration of our Future (IISECF).

Isn't that silly?

Up till now, mostly, the IISECF has materialized as silicon valley and high-speed globalized financial markets. San Francisco is beautiful, but it mostly smells like pee, and high-tech bankers commit fraud on a daily basis. More recently, we found out that the IISECF entails large scale surveillance systems. Terrorism is real, but you have a much better chance of stopping it by not committing war crimes than by spying on the entire world. These, however, are foolish implications of an internet which has hardly gotten its feet wet in the spirit-of-creativity-and-openness-and-growth for which its inventor's (humans!) are so noted in the animal kingdom.

Of course, the IISECF has meant unlimited porn, extraordinary shopping convenience, and instantaneous video communication. Not to mention the havoc that facebook has wreaked on most of our lives. Blogging is fantastic, webpages are wonderful, and Wikipedia is perhaps the greatest and most wonderful thing the Human species has managed to do (though it's unfortunate that it's so very unprofitable as a business). And then, of course there is YouTube, which won YOU Time Magazine's Person of the Year, 2006.

What is clear, in each case, is an acceleration of experience and of information exchange, which facilitates rapid discovery and entertainment. On the flip side, there is naturally the boredom and paralysis of infinity. But it is now possible, like never before, to be engaged for as long as you'd like with high quality written, verbal, and visual material pertaining to any which subject matter you can think of. The How-To will Show-You to Tie-Shoes, Make-Booze, and gear up your Rocket-Suit. There is no shortage of tales of individual humans using information available on the internet to achieve feats which would have been unthinkable by teams of experts only decades ago.

So the Means of Acquiring Information have Decentralized (MAID), dramatically. Yet Educational Institutions (EI) have plowed forward in their allegiance to the industrial past. In response, the IISECF gives you The Khan Academy and other Massive Open Online Courses. It is now possible to go from 1+1=2 straight through to multivariable calculus with a series of short videos by the world's best online teacher. There is very little you can't learn online. There is very little you can't do, online, either.

While the MAID is beautiful in her late teenage years, rebelling wildly and flaunting her perky breasts, her little brother, the MOPD, has only just figured out how to shit in the toilet. But this is a major stepping stone. The Means of Production have begun to Decentralize (MOPD), and getting your shit in the toilet is no small score.

So what is MOPD? Well, it basically started with the Grateful Dead, and the introduction of massively open sharing of recorded music. It took a few more decades for Napster to pick up the slack (just a few years after Jerry Garcia's death), and bring the means of producing copies of recorded music to the masses. Then YouTube hits, and suddenly everyone is a celebrity. The means of producing entertainment are highly distributed and decentralized. But that is only where it begins.

If the Grateful Dead are the Grandparents of MOPD, Richard Strassman is the Father. Strassman GNU (read: knew), as do many of us today, that software will eat the world. It could either devour it ravenously, or savor it, delectably, tastefully. GNU is devoted to the latter, through its commitment to open source, to sharing, to community. But software is unquestionably a means of production, and in the context of massively open online sharing of source code, software becomes the foundation around which the Means of Production Decentralize.

The MOPPPO question repeatedly comes up against the problem of dishonesty, of cheating, of lies, and of greed. It has been unable, on all accounts, to avoid it. But open source software platforms are immune to dishonesty, cheating, and lies (nothing is immune to greed), as they are open, available, and free. They embody the fundamental premises on which we would like to organize society.

The Open Source Software Movement (OSSM, pronounced, "awesome") has in turn given birth to an "Open Source Hardware Movement" (OSHM, or, "awesome" with a lisp), that brings you little $50 computers in the form of RaspberryPi and BeagleBones and is very nearly knocking on your door with a 3D printer. So now you have a webserver the size of your palm, plenty of free code to run it on, free instructions on how to do everything, and printers that'll print you parts for whatever clever little piece you think of.

I'm telling you, the MOPD and the MAID are here to clean up the MOPPPO. And most recently, that clean-up process has emerged as Bitcoin, the decentralized virtual currency with distributed computational power larger than that of the world's 50 biggest supercomputers combined. With no central issuer, no third party, no implication of military support, and no regional home, bitcoin is the worlds first decentralized financial system. She is a MAID, with a very special kind of MOPD.

Bitcoin deems irrelevant some of the most oppressive institutions in our society: banks and money-lenders. But it also lays the foundation for a host of software suites that radically change the way we interact and exchange. Just like the HTTP (Hyper Text Transfer Protocol) used by your browser, and the TCP/IP (Transfer Control Protocol, Internet Protocol), which HTTP runs on top of, used by your computer, Bitcoin is a networking protocol that facilitates exchange by ensuring the co-operation of all participants. You cannot defraud bitcoin. Just like you cannot defraud HTTP. That is not say you cannot find hacks in the infrastructure that surrounds and supports the protocol (like bitcoin exchanges and web browsers), but the protocol itself is un-voidable.

In the past, communities were bound together by religion, whose original form is the Latin 'religere', 'to bind.' But religions are often violent, emotional, irrational creatures which destroy populations and corrupt their bearers. In the age of the internet, a hundred years after Nietzsche pronounced the death of the divinity, that which binds us is no longer an arbitrary and violent belief system, but an open and honest internet protocol. Without even thinking of it, we participate in these protocols by default every time we open a browser, or spend a bitcoin. We do it daily, peacefully, co-operatively, and efficiently. It is truly a marvel of design.

But that is precisely it. Design is not a zero sum game. And it is very possible to design one's self out of any circumstance and into another. Today, the IISECF is the new flurry of protocols which will be designed and shared freely for use in the constructive, decentralized, and efficient organization of society. Bitcoin is the first of these. Facebook is the zeroth. But we can expect to become linked to each other in the co-operative organization of society in ways unimaginable to anyone seriously involved in the MOPPPO debate. This is the story of the MAID and the MOPD, and it will be very exciting to see it pan out.

And you know what the best part is? You can contribute. Just start writing code.

Sunday, 21 July 2013

A Basic Guide to Deriving Politics From Physics. First Take.

As a civilization, we happen to have found ourselves in the curious but uncomfortable situation of being positively confused as to how social systems and economies ought to be operated and implemented. The confusion is so rampant and unchecked, that students can spend dozens of years studying political philosophy and economics and yet remain positively clueless as to the real nature of the beast.

On the other hand, your average physicist, mathematician, or engineer acquires a decent set of skills, and is provided with great insight into the functional forms of things in our universe, yet is left relatively in the dark about how the tools of the physical sciences apply, if at all, to Human socio-politics and culture. Perhaps this blog post will help rectify all that. Perhaps not.

The essence of the problem is this: how are the actions of a large number of individuals co-ordinated to give rise to something which promotes the further co-ordination, well-being, and growth of those individuals - something that is wholly greater than the sum of its parts?

Let us call the emergence and maintenance of such co-ordination, well-being, and growth "sustainability"

How can we structure society, then, for optimal sustainability?

This problem, of the massive co-ordination of a large number of agents, or parts, has been treated historically by the field of physics known as statistical mechanics. Statistical mechanics set out to explain the emergence of the macroscopic properties of matter (temperature, pressure, volume, conductivity, elasticity, etc.) in terms of probability distributions over the component particles. In doing so, it would provide a new foundation for understanding the meaning of life.

But let's take stock of our footing with a simple example. In a glass of water, there are roughly a million billion billion (Avogadro's number, or 10^23) molecules of water, each buzzing around in different directions at different speeds. Yet, when we put a thermometer in the glass of water, we obtain a single reading for the temperature. What is the relationship between the distribution of speeds of the molecules and the temperature? Statistical mechanics shows us that temperature is proportional to the average speed of the molecules in the glass. Hence a connection between the microscopic activity (the distribution of speeds) and the macroscopic observable (the temperature).

Now the glass of water is a particularly simple system. More importantly, it is an equilibrium system, since the distribution of speeds (and hence the temperature) is the same everywhere in the glass. Furthermore, this means it is a minimum information system -> a system at equilibrium has no true gradients, and so maintains a minimum of possible information content. In other words, there is little you can gather about the true speed of a particular molecule from the temperature alone.

In contrast, a non-equilibrium system maintains a certain amount of information in its structure, an amount that is proportional to its distance from equilibrium. Organisms, ecosystems, societies, and economies are very far from equilibrium indeed.

Non-equilibrium systems are driven out of equilibrium by a driving signal. In the absence of driving, a non-equilibrium system decays to equilibrium. But in its prescence, the structure and patterning of a non-equilibrium system emerges to re-present the patterns in the driving signal. Perhaps this is a truism, but it has profound implications for sustainability.

The sustainability we are concerned with is inherently non-equilibrium. Our bodies and economies are driven by certain signals which maintain us and structure our behaviour. For organisms, the primary driving signal is chemical (mostly the smell of food and mates), which is ultimately a re-presentation of the sun. For economies, the primary driving signals are prices, which ultimately re-present resource availability and value.

The essential problem for a sustainable system thus becomes the accurate re-presentation of its driving signal. If an organism, for example, mis-represents the availability of food (chemical or photonic), it will burn all its resources, shrivel up, and die. Similarly, if an economy mis-represents the availability of resources and the value of things, it too will burn all its resources, shrivel up, and die. Indeed, we see much of that today.

Clearly, our economic system vastly mis-re-presents the availability of resources and the value of things. But why does it do so? Why are organisms and ecosystems so much more able to re-present environmental variables than large societies? And what is the source of that mis-re-presentation?

The answer should be clear: the source of mis-representation is centralized architecture. In a society as complex and fast paced as ours, a centralized architecture that does not control literally every aspect of every individual's life (we're not quite there yet) will not have the capacity or the bandwidth to deal with the diversity and magnitude of signals that drive our society. This isn't a political question, it's an information theoretic one. In other words, if you're still using a dial-up connection, you're going to have a bad time on youtube. Governments are connecting through dial-up in a world running on optic fibers.

Now, while politicians may be intelligent individuals, as a government, they render themselves functionally incompetent by virtue of the shear complexity of their supposed object of control. Again, it's not a matter of politics, it's information theory. This doesn't say, necessarily, that government should be abolished all together. Indeed, people will always come forward as decision makers in times of need, and if you want, you can associate that phenomenon with the word government (though I implore you not to), but the sort of quasi-democratic "elect for a term" massively bureaucratic and bloated central architecture of governments today is wholly fragile and physically incapable, regardless of the intelligence of its politicians, of addressing the problems of our modern society. Not to mention their widespread bankruptcy.

What I am saying is this: there is good reason the forest has no government. In the forest, information flows freely between agents, who act on it as they do, and experience appropriate consequences. This gives rise to a phenomenon of "natural selection", whereby those more able to reproduce - those more able to re-present environmental information - survive. But unlike the view of the old evoluitionist, who speaks in the language of "survival of the fittest", what this architecture actually gives rise to is a platform for co-operation at an unprecedented level. As much as nature is a competitive battle ground, it is even more so a co-operative assembly.

Competition arises strictly on account of the fact that resources are limited. This is the circumstance which lays the foundation for informative driving signals: signals which reflect the inherent scarcities of environmental resources (food, mates, land, shelter, etc.). In response to these signals, organisms (over evolutionary time) bounce around their fitness landscapes, "looking" for architectures that are ideally suited for the conditions of their environment and which utilize optimal strategies for responding to driving signals. What they quickly find is that by collaborating with other organisms, higher-level responses which are far more effective and sustainable become possible. Think squirrels and nuts, fungi and trees, bacteria and guts, flowers and bees.

So the insight we get from nature is this: open and honest competition breeds co-operation. And co-operation breeds sustainability. Plain and simple. No need for centralization. No need for government. Just local agents making local decisions on account of local information. That's the only way to respond honestly to the enormous breadth of driving signals we have today.

Politics, then, has been asking the wrong question. Instead of trying to promote information flow, diversity, and innovation in every day society, politics has concerned itself with inhibitting the flow of information, through insane regulations and partnerships and policies and interventions. In almost every circumstance, they interfere completely with the people's ability to respond honestly to signals (mostly, price signals), and in doing so, corrupt society, destroy widespread co-operation, and encourage greed and dishonest competition.

So the answer to the problem of politics can be found in physics - in information theory and ecology, which are non-equilibrium extensions of fundamental physics. In economics, we call it market forces. In physics, we call it emergence. In politics, we call it anarchy. But it's a beautiful anarchy, and it's the only honest solution to the architecture of society.

Now, the problem of politics has historically been linked to the problem of existentialism, that is, to the problem of being Human. But democracy, our alleged modern solution, tied as it is to notions of freedom and independence and choice, is only functional if there is wide spread participation. And surely going to the polling station has very little to do with participation. So, in a rapid paced culture like ours today, true democracy can be nothing other than anarchy, because only through local responses to local information can people be honest with themselves and their society about the value of goods and the possibilities for collaboration.

Forget Republican or Democrat. The question is, will you collaborate with me to improve both our responses to local driving signals? After all, design is not a zero-sum game.

Monday, 8 April 2013

In case you still don't know what bitcoins are, or else would like to understand them better

Bitcoin is a decentralized, open-source, virtual currency created in 2009 by a man who went by the pseudonym Satoshi Nakamoto.  The true identity of this man remains anonymous to this day - only adding to the mystique of this already mythical currency.

A bitcoin, like a dollar, is a unit of currency, which means it is a medium for storing value and exchanging goods.  Unlike the dollar, however, the value of bitcoin is not backed by a military, nor is it centralized and issued by a central bank, nor can an infinite amount of it be printed.  More unlike the dollar, bitcoin is open-source, which means anyone, anywhere, with little more than a computer and a working knowledge of C++ (programming language) can learn precisely how bitcoin operates, and can participate immediately in the network.  The full code is here  

Essentially, what Satoshi did was to setup a protocol which would allow a network of users to exchange bitcoins with one another, where the transactions themselves are actually verified by each and every user.  That is, as a bitcoin user, you actually participate in ensuring that other people aren't trying to use bitcoins fraudulently.   This has enormous appeal, because it immediately prevents the possibilitiy of someone like a large bank or government from defrauding the entire network, the way they do, on a daily basis, with the Euro and the Dollar (ie. rigged interest rates, market manipulation, etc.). 

Bitcoins are created through a process called 'mining', a term chosen for its analogy to how gold and silver are mined.  Basically, mining requires that a miner devote a large swath of computational resources to solving a difficult math problem.  In return, the miner is awarded bitcoins.  But most importantly, the mathematical problem solved by the miner actually goes towards securing and validating transactions that take place using bitcoin.  How does that work?

Every bitcoin transaction that has ever taken place is stored in something called the block-chain.  The block-chain is made up of smaller units (blocks), each of which has a few hundred transactions.  The entire block-chain is publicly available to anyone who cares to look.  That is, every bitcoin transaction is public (though the IP address and identity of the buyer/seller may be hidden).  Again, major selling point for bitcoin.  You can literally watch transactions as they come through.  As transactions occur, they are added to the most recent block.  The miners attempt to add this block to the block chain by solving a difficult math problem, which involves encrypting the transactions themselves.  By succeeding, the miners are awarded bitcoins (this is built into Satoshi's code), and the transactions in the block they encrypted are added to the block chain, validated, and secured.  The further back something is in the block-chain, the more secure it is.  This is what makes it virtually impossible to defraud the bitcoin network by pretending you have bitcoins when you don't, or by spending the same bitcoin twice (since everyone is verifying ever transaction, they will notice you tried to spend a bitcoin twice, and will only accept one of the transactions in the latest block).

The protocol written by Satoshi mandates that there should only ever be 21 million bitcoins in existence.  Starting from 0, bitcoins have been mined and continue to be mined.  Today there are 11 million bitcoins.  Roughly every ten minutes, another block is encrypted (added to the block chain) and 25 new bitcoins are created and awarded to the miners that succeeded in encrypting it.  That 25 was 50 a few weeks ago, and will be 12.5 in the not too distant future.  From there, it will drop to 0 in the year 2140, at which point all 21 million bitcoins will be in existence and never another.  At that point, miners will not be awarded newly created bitcoins, but probably transactions will take on a 'miners fee' so that miners can continue to be paid for securing the integrity of the network. You too can start mining, but you may need to get yourself some specialized hardware at this point. Satoshi was clever enough to adjust the difficulty of the math problem to math the number of miners in the game - the more miners there are, the harder the encryption becomes (and the safer the transactions become!). It is all set up so that a block gets encrypted roughly every ten minutes

This is the basic essence of bitcoin.  There are many more details, but that is enough to know for now.  Given recent events in Cyprus, the general incompetency of governements, central banks, and fiscal policy, and the ever present desire for humanity to take back control of its resources and interpresonal exchanges from Big Brother, Bitcoin has emerged as a fantastic alternative to the standard nationally issued currencies.  Indeed, as the world economies continue to implode, we can expect to see the price of bitcoin continue to rise - barring of course a great scare or other black swan on the internet scene.

There are by now numerous virtual currencies, many of which have simply ripped the Bitcoin source code and made minor changes (it is, of course, open source).  These are worth pennies right now.  A bitcoin was worth pennies a few years ago.  Last year, a couple dollars.  Last month, a couple dozen dollars.  Right now, $190.  We are literally witnessing financial history.  Whether or not bitcoin will last through the decade is anyones bet, but virtual currencies are here to stay, and bitcoin is making waves.

To get started, you would do yourself a favour by reading extensively on the internet.  Start with the general FAQ

When you're up to it, get yourself a wallet (downloading one is safer than using an internet one, and making yourself an offline paper wallet is by far the safest thing to do.   Note this method does not work for macs, but alternatives exist, like using a virtual machine).  Find someone online that will sell you coins, or better yet someone local you can meet with to execute the trade.  Your bitcoins will then show up in your wallet.  At that point, you can hoard them in hopes of a continual rise in value, or you can spend them at myriad places on the internet, including ebay and reddit.  Of course, at any time, you can sell them to someone for dollars again.  But beware, the Fed is printing trillions of new dollars a year, banks are notoriously fraudulent, and the stock market is in one of its finest bubbles yet.  There will only ever be 21 million bitcoins, there will never be bitcoin fraud (though it is possible for someone to hack into your computer and steal your bitcoins, though this is highly improbable), and no one yet knows if what we're seeing is a bubble, or if this could be a real valuation of the novel financial medium.

As far as I'm concerned, the exploding price of Bitcoin is THE correction we've been waiting for in the markets for years.  Enjoy the ride! 

Friday, 29 March 2013

Life is just a fancy way for water to get around.

Most narratives of the evolutionary story focus rather predominantly on the notion of competition, of natural selection, of survival of the fittest.  Granted, the forces of natural selection are ever present in a resource limited environment (such as the Earth).  Organisms must compete with each other for nutrients, shelter, and mates, and those most successful in competition will be most likely to reproduce.  It is a well worn story, and it has been extraordinarily important in overthrowing an ancient obsession with anthropomorphized deities and creation myths.  But this is the 21st century, and as we mature finally into the era of a technologically competent whole globe societal system, with an ever growing population and an ever more apparent scarcity of resources (and competence...), it becomes essential that we re-evaluate our understanding of the development and maintenance of sustainable systems (such as organisms and ecosystems), if we are to avoid out-competing each other into nuclear holocaust.

Organisms are, fundamentally, sustainable systems.  That is their very essence.  They operate under meager inputs of energy and achieve efficiencies unheard of in human engineering (leaving aside those engineers now turning to bio-mimicry - we'll get to that).  They can withstand numerous large perturbations or disturbances (relative to their size), and in many cases come out not just alive and well but stronger than before (they are anti-fragile!).  They are enduring and productive, and in the case of humans (or ecosystems), capable of immense creativity and profound transformations of the physical environment.  Is all of this merely a matter of competition and descent by natural selection?

The simple answer is: most definitely not.  I contend, along with a growing number of others (mathematicians, physicists, ecologists) that much of the natural beauty and profound capacity of living systems derive not from the arms-race of natural selection but from the inherent pattern-forming processes that underlie the dynamics of our universe.  Life on Earth is a Cosmic phenomenon, and don't you forget it!

See, living systems are first and foremost physical systems.  They are built out of molecules and ions, and are as subject to the fundamental physical laws of the universe as anything else.  If we are going to understand their profound capacities for efficiency, work, and ultimately creativity, we must take stock of some physics.  Now, the laws of physics are unreasonably simple - I am not being facetious.  There is no reason that physics should be as neat and tidy as it is, given the overwhelming complexity of the physical universe.  But, fortunately for us (perhaps necessary for us), much of it is surprisingly easy to understand.  Of course there is the further matter of carrying out detailed calculations, but that is essentially irrelevant (if you are merely interested in ideas), and is ultimately designated to a computer.

To understand the physics of living systems, we begin with a simple system, a hot cup of coffee on a table in a room-temperature environment.  What is happening to our system, the cup of coffee, and to its environment, the room?  Clearly, the temperature of the coffee is running down - heat is leaving the cup (we can usually see it, and it is usually very pretty) and entering the room.  Why is it doing that?  Well, the coffee is made up of (liquid) particles bouncing around in a cup, and the environment is made up of (gas) particles bouncing around in a room.  The coffee is hotter, so its particles are moving faster, and so it is overwhelmingly likely for some of them to bounce their way out of the cup, and into the room, but it is not nearly as likely for particles in the room to bounce into the cup.  So heat (the bouncing of particles) leaves the cup.  When the temperature of the cup reaches that of the room, the chance of a particle leaving the cup and one entering the cup are the same (since they are all going at the same speed, on average), and so we say the system has reached equilibrium.

If given the opportunity, all systems will eventually reach equilibrium.  This is entirely a matter of statistics.  At equilibrium, things are relatively boring.  But on their way to equilibrium, things are much more interesting.  Indeed, the coffee-cup heat is only a simple example.  Far more fascinating are the patterns in rivers, galaxies, neurons, and trees.  Ultimately, all of these designs
stem from a common origin in the statistics of non-equilibrium systems.  Ultimately, I contend, this is the essential character of life on Earth, which is only supplemented and encouraged by the forces of competition and natural selection.

So let's make more concrete steps from coffee mug to living system.  First of all, consider a situation in which we maintain the heat of the coffee, say by keeping it on a hot plate, and the amount of coffee, say by having a constant (very small) influx into the cup to offset evaporation.  Then the system will dissipate heat indefinitely (and display beauty indefinitely in the heat patterns!), as it will never be able to reach equilibrium, since we continue to drive it by providing more coffee and more heat.  Such a system we call a driven, dissipative, non-equilibrium system, or just a dissipative system for short.  Its driven because there is constant energy input (coffee and heat), its non-equilibrium because there are gradients present (temperature difference between coffee and room), and its dissipative because heat is constantly leaving the system.

Living systems, similarly, are driven, dissipative non-equilibrium systems.  They are driven by sunlight (ultimately) and nutrients (more generally), they are non-equilibrium because they are chalk full of gradients and structure and interesting patterns (which don't occur at equilibrium), and they are dissipative because they give off heat (the reason night vision goggles work) and excrete waste.  So they are very much like our coffee cup.  If we did in fact let the coffee cool down, to equilibrium, we might then say that our system has 'died', since it no longer displays interesting patterns.  Similarly, for an organism, if we deprive it of food (nutrients and/or sunlight), it will die, eventually reaching equilibrium, where there are no more interesting patterns (like lungs and brains and fingers and bellybuttons and eyeballs).

Now, this tendency for systems to decay to equilibrium, to dissipate gradients, and to degrade the quality of energy available to them, is known as the Second Law of Thermodynamics and is one of the hottest topics in physics, even to this day, despite it being over a hundred and fifty years old.  Essentially, the 2nd Law defines the sorts of processes we can expect to occur in the universe: a cup falls of a table and shatters into a hundred pieces, but doesn't spontaneously re-assemble into a cup; hot things cool down, but cool things don't heat up (unless you heat them); a drop of ink in a glass of water diffuses out until the concentration is equal everywhere, but doesn't spontaneously re-condense into a drop; and so on.  How is it then, in the context of a universe where gradients are always set to run down, that systems as complex as cells and organisms and ecosystems could possibly be built up?  Indeed, this is the standard argument of creationsists and religious folks who know only too little physics.

The key is driving and dissipation, as we noted above for the coffee cup.  A system which is driven and dissipative spontaneously organizes into wonderful patterns as it attempts, as best it can, to dissipate the driving energy.  The patterns that form in cigarette smoke are precisely these sorts of patterns - they emerge in the pursuit of dissipating the energy available in the form of burning tobacco.  The patterns that form in the Earth's atmosphere are similar - they emerge to facilitate the dissipation of the sun's energy into heat.  Hurricanes and tornadoes, which in and of themselves are highly structured patterns, emerge because they facilitate the destruction of many other surrounding patterns - ie. they facilitate the dissipation of gradients and the degradation of energy, in direct accord with the Second Law.  Self-organization in one place, then, emerges in order to facilitate disorganization in another.

On Earth, the primary system for dissipating the incoming sunlight energy is the water cycle.  The patterns of the atmosphere, which are many and quite formidable, exist because they facilitate the dissipation and degradation of the sun's energy.  If they weren't there, then sunlight would just bounce back into space with the same energy it had coming in.  But this is less likely, according to the Second Law, and so we have the formation of patterns in the atmosphere.

Living systems emerge as an extension of the water cycle, in its capacity to dissipate and degrade sunlight.  Life augments the ability of the water cycle to dissipate the sun's energy.  As a result, less sunlight is reflected back into space, and more heat is produced.  According to the Second Law, this is a favourable result.  So from the perspective of this analysis, Life is actually highly probable on a planet such as ours (with abundant water and carbon), as it complies directly with the Second Law of thermodynamics.  So, contrary to the argument of the creationist (which asserts that Living Systems, by containing so much structure, violate the second law), Life is actually a result of the Second law, since it facilitates the dissipation and degradation of the sun's energy on Earth.  Hence life is more favourable, thermodynamically, then no life.

So how do you like that?!  Life is just an extension of the water cycle.  Or, if you like, life is just a fancy way for water to get around!

At this point I think it is constructive to take note of the difference between Human engineering and Natural engineering.  Human's are concerned predominantly with the conversion of heat into work.  We burn liquid fuels, giving off immense amounts of heat (and patterns in the heat, but we mostly ignore these), and use that heat to drive turbines to produce electricity.  We know, however, and have known for as long as we've had the second law, that this process is fundamentally inefficient, in that one cannot convert all of the heat released into an equal amount of work (this is known as Carnot's theorem).  On the other hand, Natural engineering (if we may use such a phrase) is concerned with the exact opposite process, that of converting work into heat: sunlight comes in (work) and is dissipated as heat.  But in between, a whole myriad of processes occur which siphon off the energy of the sun and use it to drive the various cycles and reactions that constitute Life, before the energy is ultimately dissipated.  Somewhat paradoxically, energy ends up being stored in the system for an extended period of time, contributing to its highly ordered and structured dynamics and profoundly beautiful patterns.  In this way, living systems become thermodynamically favourable by adhering to the Second Law, and so are anti-fragile.  In contrast, human engineered systems are profoundly fragile - small mishaps can lead to devastating consequences.

I suspect that in the near future, we will learn to engineer society to derive all its energy needs from incoming sunlight, and to comply more directly with the second law, slowly siphoning off the energy for the myriad processes of our socio-economic system, storing it on board for extended periods of time, and finally dissipating it as heat.  In this way, we will become more thermodynamically favourable and ultimately anti-fragile.  We will become sustainable.

Thursday, 10 January 2013

Intro to Information Theory

Information is everywhere, but that's obvious and everybody is saying it.  In a certain sense, at the very heart of experience, there is nothing other than information.  But that's a story for another day.  So what do we mean by information?  Certainly, there is a sense of relationship, of correlation, of geometry.  But can we make it precise?

We can.  Information Theory is extremely well developed and mathematically rigorous.  Here, we will briefly explore the basic intuitions that lie behind it.  In the future, we will explore its many consequences.

Everything starts with probability theory, more particularly, with the main object of probability theory, the random variable.  A random variable is a variable which, lo and behold, takes on a random value from some predefined set of values.  The number of hours a day a person spends in traffic is a random variable, with values taken from positive real numbers between 0 and 24.  A more abstract random variable is the configuration of human civilization in a particular point in space and time.  If we 'measure' this configuration, then we find that the random variable takes on a value of, say, the exact current configuration of our society today, measured by some macroscopic qualities (population density, lifespan, distribution of wealth, velocity of goods, etc.).  But this is only one of the many possible values that the random variable configuration of human civilization could take on.  Indeed, many an author have made their fortune exploring other possible values of this wretched-though-magnificent random variable.

So the neat thing about a random variable is that it comes with a probability distribution.  That is, if we measure the random variable repeatedly, some values may be more likely to appear then others, and so we can talk about the probability of the random variable taking on any one of its many possible values.    We call this set of probabilities a probability distribution over the random variable.

Consider a coin toss.  The result of a toss is a random variable, whose possible values are 'heads' and 'tails'.  If it is a fair coin, then there is an equal probability for both possible outcomes (its probability distribution is uniform).  On any given toss, you literally have no idea what is going to happen.  It's either heads, or tails, but you can't do any better than that.  So naturally, when you flip the coin, and find out that it has landed heads or tails, you will be pleasantly surprised, because you had no inkling as to which it would have been before hand.  We might say, "the probability distribution of a fair coin exhibits maximal surprise."

Consider now a coin which is not fair, but which, by a clever trick of its design, comes up heads 90% of the time when flipped.  If you knew this, then you won't be surprised at all when you find out the outcome of the flip, most of the time.  So this probability distribution has considerably less surprise, because it's almost always heads.

Now this surprise-quantity more commonly goes by the name of entropy.  That is, the entropy of a probability distribution over some random variable is a measure of how little one knows about the outcome of any measurement on the variable.  So its like an uncertainty.  But it is simultaneously a measure of how much knowledge one can gain by measuring the variable, given how little one knew before hand.  And so in an interesting sense, the entropy of a variable, its surprise content, is a measure of its information content.  A probability distribution which is uniform is most uncertain, having maximal entropy, and the most information to gain from measurement - the most surprise.

But that's not the whole story.  When we talk about information, we almost always refer to information between two random variables.  We are not so much interested in what someone says as we are in how much of what is said is heard and understood properly by the receiving party.  Suppose there was a mysterious light on the wall that flashed colours when you flipped your coin.  You soon notice that every time it flashes blue, you flip a heads.  It's flashing blue now.  Will you be surprised when you flip heads?  Not at all.  The flashing light, by being correlated to the outcome of your random variable, reduced the entropy of your random variable.  It used to be a surprise, but now that you have the light, its not.  So there is mutual information between the light and the coin.

Mutual information then is defined in this precise way, as the reduction in entropy (uncertainty) of one variable, given the value of another.  We can theoretically measure the information content between any two variables, or more, by sampling their respective probability distributions, and calculating how much each variable reduces the entropy of the others.  In the ideal case, where there is maximal information, the entropy of a variable is reduced completely when given the other variable.  So the maximum mutual information between any two variables is the entropy of the variables themselves! (Actually, it is the lesser of the entropies of the two variables).  That is to say, something which has a low entropy to begin with (ie. you have some degree of certainty as to its outcome), is necessarily restricted in how much information it can share with another variable, compared to something which has a higher entropy. 

In most cases, this result, that the maximal mutual information between two random variables is the lesser of the entropies of the variables, has a lot to do with the number of possible outcomes of a variable in the first place.  A coin toss, regardless of its probability distribution, has only two possible outcomes, and so is obviously limited in the amount of information it can share.  But similarly in large systems, with many possible outcomes, a distribution which maintains a low entropy necessarily restricts the information sharing capacity of the system.

So there is an interesting trade-off.  Something which is well organized, about which there is little uncertainty, which is accurate and precise and efficient, ie. something which has low entropy, is fundamentally restricted in the amount of information it can share with another variable, by virtue of the fact that it is so organized and predictable!  On the other hand, something which is initially more uncertain, which may be less reliable and effective but certainly more opportunistic and versatile, ie. something which has higher entropy, is able to share far more information with another variable, by virtue of the fact that it can be found in its other states with greater probability.  That sort of trade off permeates everything.

I think about these ideas most often in the context of protein signalling in the cell, where proteins take on particular shapes and are sometimes well ordered and sometimes totally disordered, and one-way-or-another these structural differences relate to their functionality and give rise to the marvels of our existence.  But on the other hand, consider our political organization, and the chains of beaurocracy, clamping down on entropy everywhere they can with enormous energy expenditure.  The reduction in entropy is catastrophic in the face of how much information our governments need to handle if they are to do their jobs.  We need far more disorganization in our society, because only with a higher internal entropy can we properly encode the enormous wealth of information about ourselves and our environment that is necessary to keep track of in the maintenance and evolution of the (supposedly) most complex and intelligent species on the planet.  Libertarians have been saying this for a while but I think I just grounded it in physics. 

Information theory is cool.