## Antifragile: Measuring Complexity

This post forms part of series I’m doing following the 2 day course I attended in Boston. Yaneer Bar-Yam was speaking along side Nassim N. Taleb.

It might be easier to think about the simple first. What makes something simple? A square is simple. It has 4 straight sides, all the same length and at right angles to each other. Perhaps not the most concise description but you get the idea. Simple things don’t need many words to explain them.

Yaneer asserts that it is the description of an object that can be used as a measure of its complexity. There is a study that shows roughly speaking, the number of bits of information held in a sentence is equal to the number of bits required to store this information (once you take into account vocabulary and grammar). In science these kind of order of magnitude comparisons and easily made. Some may find the ambiguity worrying.

It also turns out that the media used for the description is not important. A photo or video is just as good and the information total can be calculated in the same way. This makes even more sense if compression algorithms have done their magic. I guess any data can be used to provide the description. There are clear parallels here with information theory and the idea of Shannon Entropy. A few times in the course entropy was used however because this is generally accepted to be an absolute measure based on the state of the atoms in the system it isn’t general enough for this discussion.

So when describing the state of a system, the description needs to be long enough to distinguish between all possible states.  If you have a simple system like a traffic light, the state can be expressed simply as red, green etc. Something with more possible states, like a given state of play in a game of chess will require more data. This is fine and fits well with Shannon Entropy.

What Yaneer does next is for me a bit of genius. Like Einstein who moved away from absolute measures and made things relative to the observe Yaneer does the same for the description of the system. This now means the perspective you take when looking at a system will change the length of the description required. Yaneer uses the term scale to describe this effect. Complexity of a system isn’t an absolute measure.

For example let us consider a gas. Now at one level the information required to describe the gas will be equal to its classic entropy and will be based on the speed and velocity of all the particles, high entropy, lots of data required to describe the system and so very complex. It is hard to predict where any given particle will be. However if we move away from the micro towards the macro state of the gas then it becomes simple. We have a few variables such as density, temperature and pressure required to describe the gas at this scale. It is also easy to predict what will happen over time.

The purpose of this post and the course itself is to consider how complexity forces us to reconsider how we work with organisations of people. Consider a Roman attack formation. At the small scale this looks complex and would take some discipline and training to work correctly however at the large scale it makes things simpler for the commanding officers. It is possible for one person to direct and control the outcome of a battle.

Contrast this with the way the Viet Cong fought a guerrilla war against the overwhelming firepower of the US and South Vietnamese forces. In that case the same tactics that had worked since Roman times failed catastrophically. The hypotheses given by Yaneer is that it was the complexity on the ground that meant it was not possible for a small group of commanders to give orders that could possibly result in success. When using a command and control structure the complexity of the problem you can solve is limited to the complexity the person in control can handle.

If you want to read more on this topic then Yaneer’s book “Making Things Work” is a great read.

Posted in Agile, Antifragile, Complexity | 1 Comment

## Antifragility: A User’s Manual

I’ve just returned from a two-day program for senior management run by the NECSI as part of the Executive Education programme.

The program’s strapline was ‘Learn to thrive in a volatile and complex world by creating Antifragile organizations that thrive on stress and disorder’ or put slightly differently ‘When strong winds blow, don’t build walls, but rather windmills: there is a way to turn every bit of adversity into fuel for improvement.’

The 2 days were hosted by Nassim Nicholas Taleb and Yaneer Bar-Yam.

These posts are my take on the ideas presented. There are in no way an accurate or complete and where possible I’m using examples from my own context rather than the examples given by Nassim or Yaneer.

Concepts

Strategies

• Bar Bell
• Evolution
• Small is Beautiful
• Skin in the game

## Antifragile: Black Swans

The term black swan was originally used to describe something as impossible. Back then no-one had seen such a beast. Later when one was found in Australia the meaning changed. It was then used to characterise the thing that once thought impossible is now known to be true.

Taleb defines a Black Swan event as having three attributes:

First, it is an outlier, as it lies outside the realm of regular expectations, because nothing in the past can convincingly point to its possibility.

Second, it carries an extreme ‘impact’.

Third, in spite of its outlier status, human nature makes us concoct explanations for its occurrence after the fact, making it explainable and predictable.

So why is this important? The human nature bit can be explained by the Hindsight bias it is the underlying assumptions we make about probability that lead to a different perspective on the world.

Our intuitive understanding of probability is generally based on the normal distribution (aka. Gaussian distribution or Bell Curve). Let’s take the height of people as the example. We generally expect to see the height of people to cluster around some average.

The chances of seeing something a long way from the average drops very quickly (based on something called the standard deviation). In our example of height the chances of seeing someone who is 5x the average height is pretty much zero. You don’t need to see many people to make a good guess at the average. If the normal distribution where the only probability distribution out there in the real world then we could simply position any extreme event at the edge of the curve and relate the possibility of the event back to the average (which is easily to calculate from a few observations).

It is clear then, that the Black Swan events don’t follow a normal distribution. They have a distribution where the most commonly occurring events can be easily mistaken for the normal distribution, but where extreme events can occur much further away from the average. So this time let’s consider the size of meteorites. Unlike people meteorites don’t have an average size. They follow a power distribution. Taleb called them thick tail distributions (making a point long-tail wasn’t correct). It doesn’t take much imagination to put large meteorite strikes in the Black Swan category.

Taleb then went on to characterise the two opposing environments as Mediocristan and Extremistan. Mediocristan being the normal, bell curve and Extremistan being the world of Black Swans, power laws and thick tails. Read the book to get more on these.

A good example is to look at salaries vs. heights. Say we take 100 random people from the planets population and calculate an average height and average salary. If we find the tallest person in the world and bring them into the room, the average will go up a couple of percentage points at worst. If we then invite Bill Gates into the room we can see that the average salary will go ‘through the roof’. Height belongs to Mediocristan and salary to Extremistan.

So what does this mean for IT projects? Let’s consider a standard IT task. Now depending on your project management methodology you may call this a user story or some form of Kanban card, for the purpose of this exercise it really doesn’t matter.

The size of this unit of work is based on either an estimate (using a developer’s experience) or some historical data (burndown chart or cycle time) again it doesn’t matter which. This step makes the crucial assumption that the estimated time vs. the actual time will vary based on a normal distribution. We assume it is possible to calculate an average from a small number of previous examples of similar work.

However we’re not working with standard tasks. We’re not making the same widget time and time again. This is knowledge work. Now I’m sure there is some software development tasks that belong in Mediocristan, however my opinion is that these are exactly the kind of tasks you can automate or otherwise shift away from your development team.

So we’re very much living in Extremistan, and here we leave ourselves exposed to Black Swans. Averages don’t work. Impact on project timeline is massive. You can talk about continuous improvement, flow and incremental experiments but if the data you are using to drive these decisions is sitting there with a big thick tail of outliers then you’re doomed I’ve even seen people recommend the outliers are removed to get a better “fit” for the data.

So what does a Black Swan perspective on managing a project mean for some of the current best practice?

1. Measures such as velocity will do a very poor job of predicting the future.
2. Spending time in retrospectives on the cause of Black Swans is largely a waste of effort (making something that is random explainable and predictable is human nature).
3. Games that illustrate how Scrum/Kanban work using a simple Mediocristan example only help fool us further (packing envelopes or making coffee etc.).

The impact of this insight I will explore further in future posts but for now I’ll leave you with the words of Tom DeMarco

Consistency and predictability are still desirable, but they haven’t ever been the most important things. For the past 40 years, for example, we’ve tortured ourselves over our inability to finish a software project on time and on budget. But as I hinted earlier, this never should have been the supreme goal. The more important goal is transformation, creating software that changes the world or that transforms a company or how it does business.

Posted in Agile, Antifragile, Complexity | 1 Comment

## More imagination less production line thinking please

I could pick any number of tweets over the last couple of weeks telling us how hypothesis are better than questions, are better than stakeholder stories, are better than user stores, are better than use cases are better than features, are better than tasks etc.

OK there is some merit in these thoughts. Kinda.

What worries me is the monumental lack of imagination sitting behind this constant rehashing of the same basic idea.

We get it we do. You have some work. Break it down into small chunks of work and then work on them one at a time or in a batch. It’s a production line. Introduced by Ford in 1913 (but probably pre-dating that by a few hundred years). Of course Toyota did something with their one piece flow. However the fundamental idea remains the same.

So why is everyone applying this same mindset to knowledge work or more specifically software development? I thought I’d explore the “Product Roadmap” to think about how things could be done differently.

A traditional roadmap would consist of a list of features with timescales against each. You can almost smell the factory feel to this approach. Imagine new models of car rolling off the production line. One at a time. No uncertainty in this vision.

So people are challenging this view of things. They want to change the concept of a “feature release”, to be something else perhaps a lean startup style list of hypothesis. Or framing the feature as a question to be answered. This is a small step forward for sure. But surely not a paradigm shift.

What I find ironic in this example is we have the essence of at least one different way of doing things in the name itself. A roadmap could push us in a totally different direction. Imagine mapping our knowledge about the environment in which the product exists?

This is not the pin-point accurate maps we get these days, more the sketchy mappa-mundi of days gone by. After all we now where we are and we know even better where we’ve been. The rest of the map can then explore possible future directions we can take. Perhaps sign-posting areas of interest we should consider visiting. Or the classic “here be dragons” area we should avoid.

Neither is this just a plea for more creativity. There is some theoretical basis to the shift in mindset too. Complex systems research suggest that by considering a multiple viewpoints (both at different scales and from different perspectives) this helps better understand the dynamics of the system we are considering. Also the use of options are often promoted as a way of thriving when facing complexity. Both of these can be easily visualised in map format.

So please start challenging the production line whenever the work becomes more about thinking than it does about doing.

## Sitecore MVC – A Dynamic Model

This is the second post in a series about using Sitecore MVC out-of-the-box. Last weeks post explored how to put together a simple page. This week I’ll try and create a typical navigation control which will involve pulling out data from the hierarchy of child items. While I believe this would be possible using views alone, the code would quickly become very messy, so I’m going to start by looking at creating my own model.

I’m still working with the a view rendering. Personally I think we only need to drop to a controller rendering when we have some kind of interaction with the user. Paging through a list of articles for example. I’ll want to fill the Model field shown below with a class I write myself.

To get started I need to create a class with the IRenderingModel interface. Note I’m playing with formatting to aid readability in this blog format. Here I’m just outputting a title field when the view calls @Model.Title

```using Sitecore.Mvc;
using Sitecore.Mvc.Helpers;
using Sitecore.Mvc.Presentation;
using System.Web;

namespace SunTzu.Models
{
public class SimpleModel : IRenderingModel
{
private SitecoreHelper helper;

public HtmlString Title
{
get
{
return helper.Field("title");
}
}

public void Initialize(Rendering rendering)
{
helper = PageContext.Current.HtmlHelper.Sitecore();
}
}
}
```

I’m sure the decision to push the call to the helper function inside the model goes against the grain for many (please feel free to comment below). However my design goal is to keep the views simple and focused on final published page rendering. Ideally the views know nothing of Sitecore. They contain only domain presentation logic and layout. On top of this I believe it is the responsibility of the Render Field Processors to add Page Editor magic not the helper. Finally I haven’t violated the principle of having page mark up in the model code which John West warns about.

To register this class with Sitecore I need to create a Model item in Sitecore. At this time Rocks doesn’t provide a nice Add Model option, so either go to the content editor (which does) or use Add Item and select the template /sitecore/templates/System/Layout/Model. You’ll need to add a reference to your full class name and the assembly it is compiled in to the Model Type field. Assuming my dll above is called Suntzu.dll then I enter:

`SunTzu.Models.SimpleModel,SunTzu`

Finally I add the full path to this Model item into my View Rendering. The view itself is simple. As ever we get the full page editor experience, including MVT and personalisation logic.

```@model SunTzu.Models.SimpleModel
<h1>@Model.Title</h1>
```

Now if you like strongly typed views then we’re done. However the thought of having to write C# code every time I add a field to a template or to reference parent, sibling or child items, fills me with dread. If I’m going to be doing something which requires complex business or presentation logic but when all my views are doing is traversing the content database structure I’d prefer to write one model and be done. Life’s too short.

So time to go dynamic. Using the class above as a base I now inherit from the DynamicObject and override the TryGetMember method. I know have a single model I can use to simplify the format of my view files which only requires compiling once, after that I can add Templates, ViewRenderings and Razor files as I please. Giving me a very rapid development cycle. OK the downside is I lose Intellisense and I’ve will need to deal with field names with spaces in some how.

```using Sitecore.Mvc;
using Sitecore.Mvc.Helpers;
using Sitecore.Mvc.Presentation;
using System.Dynamic;

namespace SunTzu.Models
{
public class FieldOnlyModel :
DynamicObject, IRenderingModel
{
private SitecoreHelper helper;

public override bool TryGetMember(
GetMemberBinder binder,
out object result)
{
result = helper.Field(binder.Name);
return result == null ? false : true;
}

public void Initialize(Rendering rendering)
{
helper =
PageContext.Current.HtmlHelper.Sitecore();
}
}
}
```

So far I’m really back to where I was last week. Accessing fields from within views. Next step is to move up and down the content tree. To do this I’m going to need a DynamicItem. I’m going to move my field rendering code into this new class and then get the model to inherit from this new type.

```using Sitecore.Data.Items;
using Sitecore.Mvc.Helpers;
using System.Dynamic;

namespace SunTzu.Models
{
public class DynamicItem : DynamicObject
{
protected SitecoreHelper helper;
protected Item item;

public DynamicItem() { }

public DynamicItem(
SitecoreHelper helper, Item item)
{
this.helper = helper;
this.item = item;
}

public override bool TryGetMember(
GetMemberBinder binder,
out object result)
{
result = helper.Field(binder.Name, item);
return result == null ? false : true;
}
}
}
```

And the model now just hooks into the Sitecore Initialize call.

```using Sitecore.Mvc;
using Sitecore.Mvc.Presentation;

namespace SunTzu.Models
{
public class DynamicModel : DynamicItem, IRenderingModel
{
public void Initialize(Rendering rendering)
{
this.item = rendering.Item;
this.helper =
PageContext.Current.HtmlHelper.Sitecore();
}
}
}
```

That’s the refactoring done. Now I can start adding to the DynamicItem. First the simple case of accessing the Parent.

```       public DynamicItem Parent
{
get
{
return new DynamicItem(
this.helper, this.item.Parent);
}
}
```

The view can now access all fields on the parent item.

```@model dynamic
<h1>@Model.Parent.Title</h1>
<h2>@Model.Title</h2>
```

So we’re getting there. Back at the beginning I promised the code for a navigation control. To achieve this I need the access to child items.

```       public List<DynamicItem> Children
{
get
{
List<Item> children = new List<Item>
(this.item.Children);

return children.ConvertAll<DynamicItem>
(child => new DynamicItem(this.helper, child));
}
}
```

So here is the final view. I can see already extensions are needed to check for empty fields and the template of the content items I’m iterating through. I’ll leave that out as implementation detail for now.

```@model dynamic
{
<ul class="sections">
<li class="section">

<ul class="columns">
{
<li class="column">
<li class="title">@column.Title</li>
<li class="gap"></li>
@foreach (var item in column.Children)
{
}
</ul>
</li>
}
</ul>
</li>
</ul>
}
```

Reference

http://www.sitecore.net/Community/Technical-Blogs/John-West-Sitecore-Blog/Posts/2012/06/Sitecore-MVC-Playground-Part-5-Using-Models.aspx

## Sitecore MVC – The Simplest Page

I’m interested in what Sitecore can do with Mvc out-of-the-box. I want Sitecore to pick up a request, parse the url and find the appropriate item. I want Sitecore to provide me with standard routes, controllers, models and helpers. I want to continue utilizing the existing way Sitecore automatically assembles pages (using devices, layouts, renderings, placeholders). I also want things like page editor, the rules engine and item buckets to just work.

So let’s start with a clean install (7.0 rev 131127). With 7.1 Mvc comes enabled automatically however with 7.0 I have to enabled it. Once enabled I’ll start by create a layout. In this version there is no native Mvc layout. I have to create the layout and then delete the web forms .aspx file that has been created for me. I change the path to point to the view file I’ve created (see below) and leave the model blank.

```@using Sitecore.Mvc
<!DOCTYPE html>
<html>
<title>Sitecore Mvc</title>
<body>
@Html.Sitecore().Placeholder("body")
</body>
</html>
```

So we have started gently with a simple page showing a placeholder. Don’t forget the placeholder settings item to get page editor working. However once that is done this all works fine and I get the empty placeholder shaded box displayed. The Helper syntax isn’t exactly pretty but it gets the job done. I could spend some time now implementing my own SitecoreHelper to improve matters but to do this I need to create a new page base type. Not something to worry about yet.

Next up, I’ll add a field to my basic page item template. This will hold the html title. I don’t want the page editor to kick in here so provide the disable web edit parameter.

```@Html.Sitecore().Field("title", new { DisableWebEdit = true })
```

This all feels a bit long winded. Is there a way to make the call more direct? I can move away from the helper and make use of the RenderingModel that Sitecore has provided me. At the same time I’ll push the additional @using Sitecore.Mvc.Presentation into the Views/web.config which will keep the views a little cleaner.

```@model RenderingModel
<!DOCTYPE html>
<html>
<title>@Model.Item.Fields["title"]</title>
<body>
@Html.Sitecore().Placeholder("body")
</body>
</html>
```

I’ve lost the renderField pipeline. So going to the model direct only works in this context. However for now I see no reason not to have this option available. It will be interesting as the need arises, should additional code go into the model the helper or the pipeline?

Now we have the basics of a page and placeholder. Let’s create the simplest (but most hard working) of all renderings. That is the rich text content item. I’m going for the view rendering as I have no business or presentation logic to worry about. Unlike the layout there is plenty of support for creating these simple views within the Sitecore content editor and the Rocks studio plug-in. As with the layout the only field I need worry about is making sure the path points to my view. Rocks will even link your .cshtml file with the corresponding item in the master database, keeping the two in sync.

Given my using statements are hidden away in the Web.config, my view is simplicity itself with only a single line of code required.

```@Html.Sitecore().Field("Rich Text")
```

The result is what we would expect. The full page editor experience thanks to the renderField pipeline being run from within the standard Sitecore Helper.

So there it is. Using the out-of-the-box MVC features available within Sitecore you really do have everything you need to build a basic static website displaying fields directly from the items selected in the renderings data source. This gives us full page editor experience, MV testing and access to the complete rules engine. In the next post I’ll consider a more realistic example, where we start traversing the content tree to select contextual data and I consider how to start unit testing this business and presentation logic.

Posted in Sitecore, Sitecore MVC | Tagged | 4 Comments

## Cellular Automata: Simple beginnings

I’m not sure how old I was when I first created a cellular automata. I suspect in my early teens, sat in front of a BBC Micro typing in the Basic code for Conway’s Life from a magazine. At the time I was more interested in coding fractals and the Mandelbrot set. My imagination was not gripped.

While doing my M.Sc. in computer science I produced a graphical program for running various cellular automata in my final dissertation. The focus at the time was much more on the “computer” rather than the “science”. Beyond replicating well know rules I did little to advance my understanding of these simple little programs.

Fast forward a few decades. I’m watching the family play Minecraft, which on some levels can be mistaken for one large 3D automata. I’m reading about complexity in pretty much any management book I pick up. At work I’m seeing strange behaviour in the infrastructure of complex enterprise systems. I’ve also started coding again. It was enough to rekindle my interest in the subject.

I begin by purchasing the beautifully illustrated book, A New Kind of Science, by Stephen Wolfram (published in 2002). I’ll be using some of Wolfram’s notation for convenience. However please don’t take that as a recommendation it is a controversial book for good reason.

The most basic cellular automata used by Wolfram takes 3 inputs. Each cell uses its own state and that of the two nearest neighbour’s to calculate it’s next state. My initial thought is that it may be possible to start with a rule which only requires 2 inputs. This simplification has the added benefit of allowing rules to be described in the language of logic gates e.g. AND, NAND, OR, XOR etc. The diagram below illustrates the point and shows 3 possible configurations for our logic gates.

The table below lists the 16 possible rules for the 2 input automata. Each is translate into the equivalent Wolfram rule numbers. Note each configuration can result in a different Wolfram rule number. Where applicable a logic gate name has also been given.

Rule No. Left + Self Left+Right Self+Right Logic Gate
0 0 0 0
1 3 5 17 NOR
2 12 10 34
3 15 15 51
4 48 80 68
5 51 85 85
6 60 90 102 XOR
7 63 95 119 NAND
8 192 160 136 AND
9 195 165 153 XNOR
10 204 170 170
11 207 175 187
12 240 240 204
13 243 245 221
14 252 250 238 OR
15 255 255 255

The next simplification is to constrain the space available to the simple 2 dimensional cellular automata.  Infinite space means growth can be unbounded. The simplest bounded space is made by joining together the two ends of the line. An example 7 cell universe is shown in the diagram below. Cell zero is now neighbours with cell six.

Constraining the space available to the automata has an effect on limiting the complexity. In the example size 7 space, this gives us 128 possible states. The question now is, with the reduction of inputs from 3 to 2 and with a finite number of states, can we still see aspects of complex and chaotic behaviour? Perhaps not something I’ll be able to answer one way or another in a single blog post.

Before looking for signs of complexity let’s begin with the basic behaviour of our mini-universe. Below is shown Rule 90 being applied to a size 7 space. The space is seeded with a single cell with value 1. There appears to be a short period of rapid expansion. It takes 3 ticks of our automata’s clock to reach the boundaries. After that the rule settles into a repeating pattern which takes 7 ticks to play out, after this point the model is stable. Beyond this point there are no more surprises. A sure sign we’ve left the realms of chaos and complexity.

I’ve rushed ahead a little. By selecting Rule 90 I have settled on one of the more interesting patterns. So backing up slightly here are all the patterns produced in our simple 7 cell universe. Notice how 9, 11, 13, and 15 are negative images of other rules. While 4,5,12 are mirror images. There is also some duplication with 8 and 10.

At this stage with a simple one cell seed, this leaves us with Rule 90 (No. 6) looking the most interesting so let’s see how that varies with size of space.

So at first look it appears under certain conditions we are able to produce complex looking patterns at certain sizes (7, 11, 13) and using nothing more than XOR logic gates. Given this starting point next I’ll do a more thorough analysis of the variations, however that will have to wait for another post.

## Iain Banks on ‘Miraculist thinking’

I’ve grown up reading Iain Banks. From the Wasp Factory and through his culture based science fiction, I’ve read them all, usually as soon as they are released in hard back. So it was a very sad day when he gave the statement about his terminal cancer. I read his last book ‘The Quarry’ with mixed feelings. Happy to have another Banks book to read, sadness that this would be the last. I assumed the storyline about a young man’s father who is dying of cancer was more autobiographical than it actually was. Ironically it pre-dated his own terrible news.

It is a simple book. One location, one view point, one timeline and a few interesting characters. This gives it plenty of space to explore ideas. One that really got me thinking was what one of the characters calls ‘Miraculist thinking’. I’ll let Iain explain.

‘Miraculist thinking,’ Hol says, ‘is that which assumes that only one of our ideas or behaviours – society’s ideas or behaviours, humanity’s ideas or behaviours – really needs to change, or be changed, to somehow suddenly – miraculously – make everything okay.’ ‘Such as?’ Rob asks. Hol shrugs. ‘At its crudest it’s the Why can’t we all just pull together? argument.’

Given the topic of he book, and reading the book with Iain’s cancer in mind, it is too easy to take this as a rant about whacky miracle cures, however I think that misses the point. The context is social. Iain is talking about how groups of people pin their hopes on the next idea that will solve their shared issues. This ‘Miraculist thinking’ applies to families and businesses as much as society or humanity.

For the family, who hope that the house move, or the next holiday abroad will somehow fix underlying dysfunctions. The business where the next buzzword methodology, usually accompanied by some expensive consultant, will suddenly – miraculously – result in substantive improvements overnight. It’s all part of the same thinking.

‘Everything,’ Hol says, ‘– print, radio, television, computers, digitalisation, the internet – makes a difference, but nothing makes all the difference. We build better lives and a better world slowly, painstakingly, and there are no short cuts, just lots of improvements: most small, a few greater, none … decisive.

This is the truth of the matter. As much as we want to believe, the book sellers and the consultants, there is no quick fix. When it comes to the messy, complex, tangled, uncertain problems of humanity, society, family or business, making a real difference takes time. It takes small improvements across multiple fronts, over a long period of time. So stop wasting time on the hunt for the next ‘big thing’, instead get on with the hard work of making a better world with everything you do, one small step at a time.

Posted in Agile | 1 Comment