A cycle home along the river

A tasty early morning breakfast with friends in town, and time to cycle home. A sunny day, so take the route along the river!

Crossing the road by the docks. The sky impeccable. Delicious light of being away from the tropics, strong and clear, but not overly harsh.

The dome of one of the three graces is just visible, and at the right hand edge the tower which has sucked car pollution out of the tunnel under the river for over 100 years.

I quite like the black lumpine jag into the skyline of the block of flats a friend lives in – it’s an interesting shape, and its thick darkness avoids the rough textures of variety in curtains and decoration of some of the other residential blocks nearby.

I never lived the other side of this road because it is irritating to cross when you’re in a hurry. But on a day like this, you never mind.

Cycling through the old docks of what was the busiest port in the world, and along the front past the gig and convention centres. On my first weekend in Liverpool we blagged some Eddie Izzard tickets here from some students whose aunt had given them, and they just wanted to go to the pub – whisked straight into the show.

The ferry across the Mersey is just visible at its dock. Once on a warm summer’s evening I cycled down after work and crossed on the ferry, to New Brighton, stripped to my boxer shorts and swimming in the sea, building a castle with a bucket and spade. Liverpool often has a brightness you don’t expect.

I love going along this part in varying weather conditions – you can often see and hear the waves. The shipping is always different, and the skies and water changing in colour.

Today the tide is quite far out, and it feels more like the Mediterranean. Here you can see the other tunnel defuming tower over the waters.

Whoever built this path, I’ll always be grateful. The kind of gesture a city does that everyone then takes for granted, but changes so many lives.

The one awkward part, rarely, is here by the marina. On a busy Saturday I’ve had to wait half an hour with sail boats going in and out, before being able to cross the swing bridge!

A couple sit enjoying the view. Maritime parts converted into sculptures litter the path.

I’ve never been to any of the flats down here. The area is more attractive now with the Baltic Triangle flowering with gin bars, rave clubs and hip cafes.

The light speckles, scitter-scattering brightly across the wash. You can just see the high hills across the river.

From slightly further up you can see even more clearly the mountains of a foreign country, exotic names like Moel Famau. Street signs in a celtic tongue.

On journeys home from work through Toxeth and Dingle, zagging along streets with river views, I’d watch them. Happy to know that with a drive of an hour and a bit, I could be up in them. And from there on a clear day I could see further glories – Snowdon and its friends, giants to someone like me from this island country!

A feeling about the wild, the high peaks, the grasses and rocks and views that draws.

Even if you don’t go to something, you like to know it’s there.

Nearly home. Otterspool Promenade – named as there really were once otters here. Maybe one day the dolphins will come back too as we clean the oceans!

(Fact checking just now, it turns out they came back a couple of years ago)

Up and through the festival gardens – too beautiful to have thought to stop and photograph them, so you just get the sign looking back.

There’s a hill in there with astounding views of the countryside. And over further south a rough place to run. Muddy, verdant, uncontrolled, like you’re in the countryside.

I pass up the hill through Priory Wood, a cute place where I’ve had two bike accidents, been attacked by fire crackers and fixed a tent.

My local station has a lovely Victorian building which is also a bridge spanning the rail. Trains every 15 minutes into town, almost always can get a seat.

Commuters flock in and out at busy times, congesting the ramp up from the platform.

Amazing to think that this settlement started with a church, one of the first iron framed buildings in the world, plonked in a field. Expandable because of that architecture, to meet the growing needs of its community.

Years later the station appeared, flood filling terraced houses up to the main road.

I sneak through the alleyway by the station, and arrive at my home street.

Brash in the sunshine. The details of the cast guttering, the shape of the arc of bricks above the doors… Almost 130 years old.

Cars angled on the curbs smashing the pavements. Children born to new families. Political movements grown. Lives lived – harrowing and joyful.

And the best place in the world to have my lunch today.

On using a variety of information services

In the last couple of weeks I’ve:

  • Deactivated my Facebook account.
  • Logged out of Twitter on my desktop.
  • Stopped using Google Contacts, Calendar and Photos

I’m not really sure why, but there it is, I have. I think over the last five years in excitement at the good user experience and high user adoption, I forgot why I was negative about such centralised, out of control services in the first place.

There’s something dehumanising about being subject to them. The behaviour patterns they induce feel harmful to me. They disengage me from the world.

We need to think about and improve our information systems to be good for us, empowering, productive. Just using the default commoditised, VC-funded options leaves no room for that. We’re stuck with whatever we’re given. There’s no choice, no competition, no diversity. It’s dangerously too early to have no diversity in information systems.

It used to be that there was constant innovation and public discussion and experimentation about how social groups are formed and how communication happened on the network. The nearest you see now is discussion of varying moderation policies of subreddits, which is at least healthy. More of that please.

So instead…

  • I’m posting on this blog, which uses software (WordPress) I’ve been using for 15 years, on a server run by Bytemark, a northern UK small business.
  • I use Riot for some discussions, such as #redecentralize.
  • My contacts and calendar have gone over to Fastmail which I was using anyway.
  • My photos I keep myself, on the same Bytemark server, synced to my laptop and phone with bits of string and Resilio.

In the main this costs more in cash and time for me to do. Is it worth it? That’s not really the question, it isn’t really a choice.

If you want to keep in touch, perhaps email me or write me a letter or phone me up. Or write a blog post on your own blog, and leave the link in a comment here. Join a channel I’m in in Riot and say hello to me. Invite me to join some new blockchain-based chat system I haven’t heard of yet. Invite me to the pub by SMS. Come round to my house to see me.

Off to not Tweet this. Maybe you could give me a pingback, or leave a comment!

A thousand times an ant colony of fifty times a cat

Culture is important.

Particularly, it’s important for technology which is about how we as humans talk to one another.

Telephones, computers, mobile phones, internet… They’re all about how our particular species of primate communicates. And now they’re becoming one powerful industry newly reformed.

We’re each a neural network, thrown abruptly into a chaotic world, which by sheer force and pain and amazement has cohered into the awareness that we are.

I’m amazed when my kitten, now a young cat, first goes out the cat flap[1], climbs on the garden wall, wanders off so I can’t see him. Finds his way back with a dragon fly, tortures it in the kitchen. Emotionally manipulates me with a purr and a rub of his fur on my face.

Yet in this café… Just this one small place in this one corner of Liverpool. In this café, bursting with people with life. Eating brunch, discussing business details, chatting with their loves…

Each of us 50 times more complex[2] than my cat, bound in shared experience to me. Our own thoughts, fears, illness, hope, love, astonishment, as many, as different as me and you.

The user interface we text our family member with.

The algorithm that shows us news.

The affordance of the tool we use to invite friends to our birthday.

The etiquette of when we can phone and whether we can knock unplanned at a home’s door.

The pressure of politeness, or not, in a vital thread.

Tangled bonds of trust, centralising and decentralising in a constant game[3] of tribalism evolved over a span of ages.

I’m reading a book about ants. We’re more like them than we know.

I notice that while I think I use analysis and research (say a new product to buy), most of the time I just do what a friend does (buy what they have). Which is much like how ants decide where to join in the foraging.

A powerful communications industry, newly reforming.

Do we go with the flow, let oligopolistic capital and secret states run their course?

Or do we concern ourselves with how it forms?

We’re ants.

The scents we leave on the trail, the smells we spread when we greet, the frequencies of other workers we count as we pass…

We do all the same things and so much more.

The changing tools we choose to use to manipulate information alter the rules of our herding.

50 kittens worth of cerebral cortex per organism formed into a global colony a thousand times the population of a leafcutter’s.

Humans.

[1] Itself a computer now, only letting his surgically embedded key open it. More on that another time.

[2] Is that linear? Who knows! What does that even mean!

[3] Thanks David for the article link.

A fugue on authoritarian high modernism

He listened.

The speaker started with a natural forest.

Slide of natural forest

A gorgeous, maddening, complex, survived-of-the-fittest, multi-species, layered, organic feedback pathways, mycella-ridden delightful speckled sunlight happy, violent/eating/parasiting/singing insects/birds/people, robust kind of a forest.

They followed with a timber forest.

Slide of timber-producing forest

A straight-laced, single specied, splatted down, polygonally-bounded-tree-age-contoured, money spinning, automation hugging, zen simple, database-like, timber-producing, silent, vulnerable all-lined-up-in-rows kind of a forest.

The for loop iterated.

He wept for twenty minutes.

The peace to know what you don’t know

I avoided statistics at school like a rash – filled myself up with super powered double A-levels in Maths And Mechanics instead. I’m not sure why, as it turns out statistics is vital to the good functioning of our world in 2017.

We have plenty of computers and plenty of data, and plenty of willingness to make evidence based decisions. Alas it is easy to fool yourself into thinking you know something you don’t. For your confirmation bias to convince yourself you’ve found evidence, so you don’t feel guilty when you report up your management chain.

You don’t need to juke the stats – that is to say, consciously interfere with data collection on a large scale, as described in this scene in a school from The Wire.

No, no such outright deceit. It’s easy instead to “rig” the stats – whether by accident, on purpose, or with different parts of your mind doing different things without full understanding.

Having spent some more time attentive to statistics this year, here is a high level check list of things to watch for:

1) Make sure the sample is large enough. If you don’t have enough data, you get an essentially random result. It’ll change with each new data point. It’s easy for non experts to gather data until the apparent result hits what they expect, and then stop. Instead, learn how to calculate confidence intervals. Fancy Bayesian probability distributions can clearly show the limits to your knowledge caused by lack of data. The hard bit is the cost – the sample sizes needed to actually reach valid conclusions can be quite large. Or maybe just be honest, accept there may be simply no way to get enough data.

2) Choose your measure first and stick to it. It’s super tempting to measure several things or tweak the weightings between different contributors to an index, until the answers meet your preconceptions. Instead, choose a good measure up front and go with what you get. Overfitting models or overtraining neural networks feels like an extension of this error.

3) Don’t confound your variables. The real cause may be one thing, but it can look like it is something else which is correlated with that cause. For example, it’s easy to show that black people in the US commit more crime. You just have to ignore the confounding fact that actually it is poor people who get into circumstances where they commit more crime, and that black people tend to be poor. This one goes to the very heart of causality and meaning.

4) Classify things instead of ranking them. Often reality doesn’t neatly give things a score, but does have categories or outliers. For example, you might not have enough to exactly rate hospitals, but you do have enough to know (thanks Anna for reference) which are 3 standard deviations from the normal, so should be inspected.

I’ve noticed that it is common for highly trained, experienced people to spot one of these four things, and focus on improving that or bringing the doubt caused by it to everyone’s attention. That’s a mistake, as to come to a sound conclusion you have to follow all of them, and a lot more (for example, I haven’t mentioned accounting for measurement errors). One won’t do.

This at first can all seem discouraging. It often turns out that, despite apparent data, you know nothing or little.

Don’t despair, for two reasons.

i) Despite the above, statistics still finds nuggets. Truths you can act with confidence on.

ii) It’s valuable to know what you don’t know – to know when, despite your intuition, there really is no evidence either way for something. That management time is best spent either gathering more data, or doing something else entirely.

The peace to know what you don’t know.

Email review #1: FastMail

I’m personally still frustrated with the email services I know about. I’m going to start by reviewing what I use now, which is FastMail.

This is for my personal email, which I use for identity on many web services, for notifications and for writing letters to people. I’m working for myself at the moment too, so I use it for business in that context.

As background, be aware that I don’t think the email protocols (SMTP, POP3, IMAP) cut it any more. So I’m not splitting my email up by rating apps separately from service. I think the two are integrated together. This is because the user experience is better – setup is simpler, and unfortunately basic features like search, spam and filtering have to have a proprietary interface as the standards aren’t good enough. I’d love better protocols to be adopted on a wide scale, but realistically they’re not yet.

(In practice, I personally still use SMTP and IMAP because I sometimes like to read email in a command line client called mutt, and I still use POP3 to make my own personal backup of mail. Overall, I expect a good, modern email service to have its own nicely made app for common platforms that just works, and to support standard protocols for uses like backup.)

For clarity, I’m not going to consider GMail – the service, or the proprietary Android app – or mention it again. It has more than a billion active users, so by any sensible definition it is not just a monopoly in the UK market, but a global monopoly in a global market. Monopolies are single points of failure, and always lead to price gouging and regulation. That isn’t good for anyone – even more so as the price in this case isn’t paid in cash but in privacy and lock into other services (e.g. Google Now only works with GMail).

I first used FastMail as a second emergency email address when I went travelling in 2002. It had been well reviewed, I think on the venerable, and still apparently good, EmailDiscussions.com forum. At some point I paid for it, then they had an outage for three days, and I switched away from them. Much later, I had the privilege of meeting FastMail founder Jeremy Howard while he was at Kaggle, he’s a great person. Following the Snowden revelations, I switched back to FastMail and started paying them. In short, I’ve a long history with and like the company. It has a good reputation.

Usability 3/5 – The website is excellent. The Android app is decent, but doesn’t support offline reading or sending of mail – super important when travelling. Even when I have Internet the app is often slow or fails to connect to the server. I’ve tried alternative Android general purpose mail apps, but they all either didn’t set up (amazingly none of them had a FastMail config option, I had to type in the server names, it was tedious and still usually didn’t work) or had a worse user experience in various ways (e.g. no spam button). I don’t use the calendar or address book – I think because of the tedium of configuring them. I expect the Fastmail Android app to just sort that out for me, which it doesn’t.

Delivery 5/5 – Seems very good to me, no problems sending or receiving mail. Their spam detection is good enough – I get a few spams a week (my email has been public for about 17 years, so receives lots of spam), and haven’t had real mail classed as spam for a long time that I know of. I have to point my domain MX records straight at FastMail, and I had to consciously tell it to train a Bayesian filter just for me. SPF and DKIM were easy to set up and work.

Privacy 2/5 – Basic stuff is fairly decent, they use on the wire encryption for all protocols, and I’m confident (without having checked) that the company is well run in terms of basic security policy. Alas, while FastMail has some level of app-specific passwords and two-factor authentication, they’ve implemented those features very eccentrically. Worse of all, there is no attempt at encryption of your emails at rest (on their servers) (Correction: FastMail say it is encrypted at rest, see comments) or any end to end encryption such as PGP – they give good reasons for this, but nevertheless it makes it not private.

Decentralization 1/5 – The company is very centralized – all the servers are in the US (Correction: FastMail also have servers in Amsterdam, although you can’t choose where your data is, see comments for details), even though the company is Australian. The code is all closed. Not only is this a single point of failure, but it is a particularly dangerous one as the US is a single point of failure for lots of modern IT systems in lots of ways.

Potential 4/5 – Their business model is sound, ranging from $3 to $9 per user per month. They have both individual and small business customers. There’s no free plan, it’s just a free trial. They’ve been around for 18 years, so seem a good stable choice. They are slowly but surely pushing a new standard email protocol called JMAP, if only it had wider adoption.

Total score: 12/20

A short walk around Vote Leave’s software VICS

The campaign which won last year’s referendum for the UK to leave the European Union immediately dissolved, and in the process kindly open sourced some of its innovative canvassing software VICS – blog post, github repository.

This afternoon, prompted by a chat with Incubator of Change (a Liverpool based think tank working on new ideas for how to develop policies), I took a quick look at the code. Rather than lose what I learnt, I’m just throwing it up in this blog post.

This post is not meant to be a detailed or accurate account of the software, just an hour spent quickly scanning it. Pleases correct errors or add links to better analyses of it in the comments!

According to the blog post linked to above, the features Vote Leave needed that weren’t in existing software – and meant they had to make their own – were:

  1. Live canvassing. I think this means it is a modern web based application, updating in real-time with new data and analytics. This requires a nice web interface and a permissions system for all the canvassers and local organisers to only have access to update or read data they need.
  2. Sophisticated modelling – done by proper physicists who know what they’re doing, using machine learning. This was used to inform all sorts of things. e.g. content of leaflets sent to every household
  3. Facebook adverts – targeted in detail in a sophisticated way. They bought over a billion digital adverts.

VICS isn’t a complete system. e.g. I’ve just checked, and it has nothing for generating Facebook adverts. I think VICS is just the core data entry and permissions system – i.e. the “Live canvassing part”.

The actual voting intention data is stored in another system (PAF) we don’t have the code to – as is the stuff that does machine learning and the stuff that does social media adverts. I could be wrong, this is just first pass looking at the code.

Nevertheless, this dump lets you learn a reasonable amount about their software and process. Below are some files to look at – even if you’re not technical, do click through. I’ve tried to choose ones you can learn something from anyway!

tools/db/create.sql – the schema for their database as to what is stored in VICS. It just has wards, constituencies and users (as in canvassers etc) with permissions for what they can access. No voter data.

paf-client/src/main/java/com/infinityworks/pafclient/PafClient.java – the functions for calling what they call “PAF”, their core API, which itself is not in this code dump. PAF is where the actual voter and intention data is stored. It’s a fairly basic API for getting lists of streets, voters in streets, and storing voting intention and other data about voters.

paf-stub/src/main/resources/json/paf-voter.json – this is a fake version of the upstream API for use in testing. This file is an example stub voter – it’s useful as it lets you see the data PAF contains about a voter. As you can see it is fairly straightforward, what you’d expect in this app. Note e.g. “wants_postal” (postal voting), “lift” (to the polling station), “intention” (how they intend to vote) etc. And “issues” being “cost”, “sovereignty”, “border” (key 3 issues for Vote Leave in the campaign). Plus “volunteer” – they used this to capture new volunteers. And, of course, contact by canvasser history. Oh yes, and this test voter has, amusingly, a French name – John Deaux.

web-client/app/views – HTML templates to let you see what each screen of the end user application contained. Maps of constituents, canvassing input forms, get out the vote (gotv) and so on.

web-server/src/main/resources/pdf/canvass_cover.pdf – hand made cover sheet for canvassers, telling them how to canvas.

web-server/src/main/java/com/infinityworks/webapp/clients/pdfserver/PdfClient.java – this code makes three PDFs for printing, by calling another API. They are for 1) canvassing, 2) get out the vote, 3) labels to get out the postal vote (links are to the forms for choosing the streets and so on).

web-server/src/main/java/com/infinityworks/webapp/rest/dto/LeaderboardStatsResponse.java – there’s a leaderboard of the best canvassers, wards and constituencies. I know from running various crowd sourcing efforts at mySociety that such league tables can be catnip for the best performing individuals.

Please do leave comments with other interesting things about this software you spot, or links to better blog posts or analyses of it.

Hindu holy mountain #2: Arunachala

In the late 1890s – the story goes – an ordinary boy in southern India heard an uncle mention the name of this holy mountain.

Arunchala!

Just this, just the name, started pulling him to it. The mountain is meant to be, literally, an incarnation of the deity Shiva.

Then, a while later, for no apparent reason, the boy had a sudden spiritual awakening on the nature of death. In final frustration during some pointless grammar exercises, he left home, headed for the mountain. He stayed there for the rest of his life.

Last month I arrived at the town next to the mountain. Tiruvannamalai!

I found, more than a century later, the flourishing ashram that the boy had started (full biography). Popular with Indians and Westerners. There are meditation halls and a cow farm.

I stopped for a while by this doorway, to listen with everyone else casually, peacefully, seriously to a tape recording of tracts by Sri Ramana Maharshi, the boy’s name when he became a guru.

The tracts were giving details of a Hindu mythos. Quite intelligent, fairly abstract and almost Buddhist in their nature. Some Hindu permanent soul details I didn’t take to. Well told, easy to absorb as they were the answers to questions Westerners had asked Ramana in person.

The next morning I woke well before dawn, too early for the main town temple that I had come for to be open. As if drawn by forces unknown to me, I started to climb the mountain. It rises 2617 feet above sea level. A guide appeared to help me.

We climbed all morning. I’d been ill. I was shattered.

We emerged to this view – the mountain stood proudly alone.

I helped this man rub milk into the rock here. I’d no idea why.

Bare foot, my guide and I made our way round to a tiny cave. Puja (prayer) paraphernalia had been left there – we left a few rupees to pay for refreshing it.

Yellow powder to daub on my head and rub into the lingam.

An oil lamp to ritually light.

A lingam of Shiva – the mountain/god – to ponder.

I sat a while cross legged, taking in the visceral, messy feel of these objects. My guide left me a while, he wanted to have a smoke. I fell asleep I think, or jumped some mental state, suddenly jolting awake.

Later I found that the ancient texts say

All stones in that place Arunachala are lingams. It is indeed the abode of Lord Siva. … to fall asleep there is to be absorbed in samadhi, beyond the mind’s delusion.

Back outside, this man had finished making his puja of flowers where we’d put the milk. We paid him a few rupees.

In the only hut up there, we had tea cooked on a fire.

I bounced down the mountain, now beamish. I don’t know why.

I paid my guide (using a note a friend, during a conversation about meditation, had given me to help me out with demonetisation – thank you!). He turned out to be a sculptor. I left him, carving an image of Ganesh.

I believe nothing, Hinduism has no draw to me. And yet… I love these holy mountains.

Rant on the failure of news media

Last night I watched Charlie Brooker’s news recap of the year. As I watched, the humour leavened things and left me entertained. But in my sleep, my mind has realised it had made me pretty angry. So a brief change in tone for this blog – a rant!

Charlie – both you personally, and you as a representative of the media – stop whining about the news bubble! You collectively all made the news bubble. Take some responsibility for it!

It’s cute and funny and post-modern to take the piss out of yourself for thinking the referendum on leaving the EU wasn’t a serious political event all year as remain was bound to win, and to take the micky out of yourself for going to bed on both Brexit and Trump’s election nights assuming they’d lost for sure when actually they won…

Yes it’s cute to be self-deprecating about that absolute failure of the main middle class intellectual UK media reporting. But it’s also pathetic.

Brexit

The news was absolutely useless at explaining Brexit to me as it happened. It didn’t help that the campaigns were useless too, and indeed many of the supporters of both campaigns who I knew. But they aren’t meant to be professionals reporting on the world for me and explaining how it works. The news industry that the BBC’s recap of the news year is representative of – that was their one job.

There seemed to be no understanding and no desire for analysis as to what was going on with the referendum. You’d think I might have read about it in the Guardian or at least in the Economist that I was paying for. But no, I had to rely on things like Simon Wardley’s blog or on my uncle sending actually good academic reports on the labour market effects of migration to have any idea why intelligent people were voting to leave the EU. I had to go off and read the Daily Express all by myself to gain any kind of understanding of the general mood amongst half the country.

So no Charlie, it’s pathetic apologia for your industry to just giggle at yourself for this disastrously bad failure of your industry’s reporting on the key UK political event of the year. It utterly failed in the worst possible way. Admit it explicitly, not just in jest, and do the work to not be so shit any more.

If you were surprised on the morning of the referendum result you just weren’t paying attention. Not only did some polls get the result right – all polls were so close and variable, you didn’t need to be looking hard to know there was a decent chance that leave would win.

You can’t just look at the average poll even if it wins by a tiny amount and assume that means that will be the result. It is probabilistic! If you don’t know this no you’re not funny or cute, you’re just being wilfully stupid in a destructive way. You’ve no basic idea how polls work, they’re just amusing numbers you sometimes share on social media.

Worse, you’ve no idea how to prepare your mind for multiple outcomes. You’re trying to use the news to reassure yourself, rather than understand reality. It’s pathetic. If you were surprised on the morning of Brexit, you need to completely alter your information diet and your approach to understanding the world.

All that applies to any politically aware citizen. It applies a hundred times more to someone like Charlie in his role representing the establishment media by doing the main news recap of the year.

Trump

This was even worse. The reporting on Trump all year was unbelievably bad. It consisted of finding tiny tiny moments when Trump said something that can be interpreted as bad, and only reporting on those in a one sided way, not realising why doing this increased his poll ratings. Watching him from a distance gave you no idea why tens of millions of people would vote for him.

I had to read Ian Welsh’s blog to find anyone to explain to me how Clinton was representative of neoliberalism, and both candidates were equally bad. And then, even though he’s a deeply problematic person in lots of ways, I had to resort to reading Scott Adams’ blog to understand the persuasion techniques Trump was using to win.

Why was this material not clearly explained in my Economist subscription, or the Guardian opinion pieces I was reading? Utter failure of professional media. I had to rely on a crowd funded social media consultant and an alt-right cartoonist for actually decent quality information. Seriously!

And no Charlie, it’s not funny you were surprised Trump won on election morning. It just means you were an idiot. No, polls have not been wrong recently, you just don’t know how to read them. For example, 538 (an obscure source to use in 2008, but hardly in 2016) were giving Clinton a 65% chance of winning. I’m going to put this in bold for people as it seems a common error:

When somebody has a 65% chance of winning it does not mean they have won!!!! It means there is a 2 in 3 chance that they will win. You roll one dice and have to get a 1, 2, 3, or 4 for Clinton to win. Actually pretty good odds for Trump when the prize is the presidency – pick up a dice, try to roll a 5 or 6! Feels possible, right?

So unless there’s, say, a 99.99% chance of victory going one way in an election, you shouldn’t be surprised whichever way it goes! Either could happen.

If you woke up after the election surprised Trump won, then you’re just an idiot who needs to understand basic probability. I’m sorry, it just isn’t an acceptable reaction to have been surprised. Learn not to have it again.

I expect better from our best people hired to work for our best public service broadcaster. It’s frankly embarrassing.

Summary

So, to summarise, on both of these supposed election upsets of 2016, which have made everyone in the UK college-educated bubble claim it was a terrible year, there were two failings in basic reporting by our media:

  1. They didn’t investigate properly why so many people were voting the other way to the way they expected.
  2. They didn’t understand basic probability, couldn’t read polls, so were surprised at the result.

The two are linked and reinforce each other. The reason I knew I had to go and do the research on why (1) myself was because the mysterious polling (2) was clear.

Failing in this way is excusable for a normal member of the public, we can hardly be expected to have the time do to this work. It is absolutely unacceptable for people working in the media. If you work in the UK media, and you were surprised by either Brexit or Trump on election morning, then you need to go back and retrain.

I’m a computer programmer, not a professional media worker, and I knew. Why didn’t you?

Disclaimer

For the record I voted Remain, but maybe there are potential opportunities from Leave in a ten year timeframe (it can’t be judged before at least ten years have passed, don’t even bother trying). I don’t endorse either Trump or Clinton, and I don’t need to as I’m not American. This blog post is in rant form deliberately, so do make me add nuance in the comments if you feel the need.