Book review : Flashboys - Michael Lewis

Michael Lewis is a great writer in my book, the ability to find an interesting story to tell in the first place is a decent trick, but then to research it, and distil a complex story to make it readable and entertaining for the average bloke is a real skill. I read Flashboys over a couple of days on holiday, and had trouble putting it down.

It’s not a subject that I know about High Frequency Trading (HFT), although working at the slower end of financial services I've come across people who left my teams to move into this space, and been peers of some of the banks that are mentioned in the story.

Flashboys is essentially opening the veil of secrecy between big Wall St. Banks, their dealers, and high frequency traders. It starts in 2007 with a new fibre optic line between New York and Chicago, which cuts 3 milliseconds off there round trip time. It cost $300m to install, and made about $3bm in sales, so that HFT could buy and sell stocks quicker that other traders and make a profit without taking any risk (many HFT’s boast about never making a loss, which is no great shakes when you can see the other players card, and you’re not actually taking any risk).

It’s a great expose of a very murky world, where everyone was making money (bank, dealer, exchange and HFT) and the poor general public were losing out; it was as Lewis describes a tax on dealing that the HFT companies were making. Having read this much I was intrigued as to how the main character in the story (Brad Katsuyama a trader a Royal Bank of Canada ) would try to fix the problem, in the end what he did was to set up a completely fair exchange where there was no selling of data to competitors, no deals to buy or sell on the exchange, and that they (counter-intuitively) deliberately slowed down all access to their new exchange so that no-one could gain an unfair advantage. This made it fair to everyone and becoming (I think the book was first published in 2012) a successful exchange. I find the idea that they slowed it down by coiling up 38 miles of fibre somewhat unusual but there more to that story than is in the book I'm sure.

What makes the book interesting it that it’s written around about 10 main characters, most of whom get a decent back story so you can empathise where they came from, and why they were motivated to do what they did (some interesting links to 9/11 in there as well). There’s also quite an extensive piece on a Russian coder who worked for Goldman Sachs and was arrested for storing source code in an open code repository. What’s clear is that his case was the blind leading the blind, and a stunning miscarriage of justice, which puts the fear of god into me about the american legal system (I've not got any confidence that ours would be much better).

What I guess is most surprising is that this could be a complete fictional tale, but it isn't, it’s all true and rather scary. Overall a great read, and if you've read Liar’s poker, Money ball or 'The new new thing' you’ll really like this (even if you haven’t you will, and then read these others!)

Read More

Challenges of securing cars

Interestingly Autocar's Matt Prior has just written on the subject - here

Following on from my earlier post about the challenges of securing the software used in cars, here's my thoughts in more depth:

The car just becomes another piece of software to maintain - You may well have seen a number of stories in the press recently about software in cars being hacked.  The biggest story was the one affecting Jeep, needing to recall 1.4m vehicles (link).  The details of the hack are scary, in that the security researchers that found the vulnerability didn’t need physical access to the car, just a connection to the Sprint mobile network, and then could target any of the 1.4m Jeep vehicles.  It’s staggering to think this level of security testing wasn’t undertaken by the vendor.

However Jeep weren’t the only company found out recently.  Ford (link) and BMW (link) have also had to release security patches.  What concerns me most is; what appears to be happening to the car industry now is what happened to the software industry about 15 years ago, when Microsoft was in the press almost weekly with reports of it terrible software security.  Back then in 2002 Bill Gates wrote the famous ‘Trustworthy computing memo’ which stated “… if we don’t do this, people simply won’t be willing - or able - to take advantage of all the other great work we do,”.  Microsoft took the unprecedented decision to stand down all 9,500 Windows developers for 8 weeks to focus purely on security. (1,300 man years of work). 

However with car manufacturers having to integrate diverse vendor’s software (infotainment, ABS, Steering, Throttle, climate control, stability, traction control systems, all of which may be written by different suppliers), not only so they work safely (although that doesn’t always work - see this long and very scary article on Toyota which essentially says it wasn’t even possible to test if their software was roadworthy).  Alongside the challenge of integrating disparate software, which is certainly not a skill most car makers will have extensive experience of, combined with needing to think about security at all stages, rather than just testing before the car gets released, is a huge mind-shift change, and one illustrated by the Microsoft example that takes time and huge investment, neither of which the car industry has.  The final danger is that the motor industry is a very marketing and consumer demand led business, so I can easily see a situation when ease of use for the customer trumps security, and weaknesses are consciously built into systems, making them far easier to attack.

What can be come about this? well it will be interesting to see how the industry evolves.  Tesla, so often the darling of the car industry was also shown to have vulnerabilities this week, but this required physical access to the vehicle which is much less of worry.   Tesla are in a better position as software was designed in from the inception of the vehicle so the design team are already thinking about security from the when the car was on the (virtual) drawing board.  I wonder if traditional car manufacturers can keep up with the software development world, This article on LinkedIn (link) illustrates the number of lines of code in a modern car (a staggering 100 million), which is greater than the combined lines of code of a Boeing 787, the space shuttle and Windows Vista, which I find a terrifying statistic.

Tesla are turning this to their advantage.  A recent tweet from Elon Musk their charismatic CEO told owners of their P85D cars “0-60 acceleration time will improve by ~0.1 sec soon via over-the-air software update to invertor algorithm”.  While quite cool that you get additional performance ‘for free’, and they are currently talking about adding a collision avoidance feature in a future update, it does ask an interesting question of insurers. 

Historically when you bought your car the features stayed constant over its lifetime, so insurers needed little data beyond make, model and year.  However with the concept of over-the-air updates how will insurers price car insurance? Will you need to declare that your car is running the latest firmware or to have to guarantee that you will keep in updated within 30 days of an update being released ?  And as we all know from updates to our mobile devices you probably don’t want to upgrade on day one in case there’s a bug in the software which renders your car un-driveable.  This is certainly a topic that's going to gain many more column inches over the coming months.

Read More

Delivery Engineering Team

Interesting to read Greg Dziemidowicz's Blog (link), thanks to the great DevOps weekly from @GarethR, about how he runs a Delivery Engineering team.

Image - Greg Dziemidowicz

Where I work we setup a Platform Services team run by Jon Fletcher. There's a big disconnect in my view for anyone how has a DevOps team.  We provide a DevOps service, so we help others get better at DevOps concepts, but its not the teams job to do the releases, we help others get better at theirs.  In my mind if you've got a DevOps team doing releases you've just added a silo rather than trying to break them down.

I like Greg's definition of what his team does :

" Delivery engineering team enables others to deliver business value faster "

I was also interested to see his team are responsible for helping reduce new developer laptop setup time, this isn't something our team gets involved in, but it's a fairly easy extension to the teams existing responsibilities.

I think it's equally important to be clear on what's out of scope, the team doesn't do release support, but as they own the DevOps platforms (Puppet, Jenkins, Octopus etc.) they are responsible for support of the platforms that our other teams use.

Read More

FinTech - marrying up finance and tech start-ups

There's lots and lots in the press, and the word FinTech is certainly getting a workout (too much many would say).  This Bloomberg article caught my eye, not only as it's a good example of a bank doing it's own incubating and working directly with start-ups, but because of the synergy between the two companies.


  "Wells Fargo’s investment in Context360 is part of a technology incubator program the bank began last year to spur innovation. In April it selected the San Mateo, California-based startup and two other firms out of almost 300 applicants for one-time investments of between $50,000 and $500,000."

"Customers may be more receptive to offers or notifications if phone sensors show they are commuting by train, watching soccer practice or otherwise in a place where they can look at their phone. The results could theoretically be used by Wells Fargo to tailor alerts that would hit customers’ phones just when they are engaging in an interaction that might require the bank’s products -- like a car loan.

Or it could be used to detect fraud by checking that a phone and credit card are in the same place, and stopping a transaction if they aren’t."

Read More

Google's nearline storage announcement...

From a sheer scale perspective this new announcement from Google is crazy...

They've announced their new nearline storage offering with a staggering 3 second restore time (rather than AWS's Glacier at hours restore time), Google nearline cloud storage is also based on flash and disk rather than AWS's blu-ray.

To launch the product Google are giving customer a free 100 petabytes of storage for 6 months !

as the Reg article says, it's not AWS that will be worried, but the tape vendors, soon there will be no reason to use tapes for backup as nearline storage will be cheaper and simpler...

Read More

Hacking cars

There's been lost of news in the IT press recently, which even tipped over in to the FT at the weekend, about hacking of IT systems in cars.  The most well publicised article was this one in Wired about a Jeep.

What's really terrifying is the complexity of systems within cars.  This article on LinkedIn talks about cars having more lines of code in them than Windows Vista, the space shuttle and a Boeing 787 combined.

With that much code, and probably from a multitude of suppliers, no wonder it's a challenge to keep them secure.

This all smacks of the early days of Windows security (or lack of it), it took a number of years for Microsoft to get good at it, (which is it arguably now).  However Microsoft own all their own code, and have the resources to get it right.

For car manufacturers, this is a big deal, a non core skill, and something that they won't want to focus any more time and energy that they absolutely have to.

Where will this lead ?  Certainly many more vulnerabilities and headlines of compromised cars.  Alongside this as the battle between adding features & usability and security continues, with car makers adding 4G and Wi-Fi there's a rocky road ahead.

Will we get to stage where we have a Ford 'Patch Tuesday' and all owners will need to upgrade their cars ? Will you want to be the first customer to test a car makers patch ? With the range of disparate systems and manufacturers in your car, will that patch work first time and flawlessly ? If big manufacturers can't get it right on a mobile phone, what chance on a car with far more code ?

It's not all bad news, Tesla is at least using software updates to add more features, how about a fast 0-62 mph time ?

Tesla P85D 0 to 60mph acceleration will improve by ~0.1 sec soon via over-the-air software update to inverter algorithm

— Elon Musk (@elonmusk) January 29, 2015

Read More

Immediately available DevOps Engineer

Few things make my blood boil than a recruiter spamming me with 'DevOps Engineer'CV's, just because someone's worked with AWS and uses Puppet and whatever else doesn't make them a DevOps engineer (I don't even believe there is such a job title), what that person is, is an automation expert.  If they were some sort of DevOps coach or similar, their CV would have much more focus on the cultural change, or working with both Dev and Ops teams, not software automation.... Rant over

Read More

Open container project

We've not delved in to containers at work yet (still working on walking before running), but already some of our key suppliers are talking to us about providing future versions of their software as containers.

Therefore we'll need to get on that journey, and with all these types of emerging technologies, choosing the right horse to back is always a key decision.  Docker has clearly led the market, and is a real buzzword at the moment, but with the recent play Microsoft have made with the Microserver announcements, the market is becoming more crowded.

This all got more complicated when some of the vendors (Docker) started talking about different standards, which was never going to help the consumer.

So the announcement at Dockercom yesterday (neatly summarised by TechCrunch here), where all the big players (Docker, CoreOS, Google, Microsoft, VMWare Amazon etc.) will all play nicely together, with docker donating their container format is a massive boon for us consumers.

It was also interesting to note that the odd one out (it seems to me) in the Open Container Project is Goldman Sachs, everyone else is a technology company.  Interesting move for a bank, but clearly one that thinks of itself also as a technology company

Read More

US Government OPM hack

Interesting and rather scary insight into another big US government security breach, the OPM (Office of Personnel Management) has been hacked, and large numbers (up to 14 million) of government staff's data stolen. The article talks about how until 2013 they didn't have anyone in an IT security role, see the quote below...

" The OPM had no IT security staff until 2013, and it showed. The agency was harshly criticized for its lax security in an inspector general’s report released last November that cited its lack of encryption and the agency’s failure to track its equipment. Investigators found that the OPM failed to maintain an inventory list of all of its servers and databases and didn’t even know all the systems that were connected to its networks. The agency also failed to use multi-factor authentication for workers accessing the systems remotely from home or on the road."

It seems incredible that a government agency has been so lax even in the last 2 years, I hope the UK government is taking better care of its staff's data.... (that might be a big hope)

http://www.wired.com/2015/06/opm-breach-security-privacy-debacle/

Read More

Facebook hardware

You might of heard of OCP, Facebook Open sourcing it's hardware designs (Facebook righty state that hardware is not a competitive advantage to them), so making it freely available to anyone doesn't hurt them.

They started doing this around 6-7 years ago and now it's starting to become mainstream.  A couple of points in this article jumped out at me though...

Firstly that  "Goldman Sachs has committed that "80% of its servers would be OCP." It will only buy regular commercial servers for oddball, outlier applications, where it doesn't make sense to spend time on custom designs.

And secondly that HP is now becoming an OCP partner, and will build a range of OCP compatible hardware, which is cannibalising their own server line up, but it's an interesting move by HP, and one to keep an eye on.

Full article here : http://uk.businessinsider.com/facebook-open-compute-project-history-2015-6

Read More

Ranger 4 DevOps day presentation posted to Speaker Deck

I've just posted the talk I have at Ranger 4's recent DevOps event at 1 Aldwych.  It's similar in content to the presentation that I did at IBM's Interconnect event a couple of months ago, but is a little longer and more in depth on the journey that Hiscox have been on, in our march towards DevOps.

The session seemed to be well received with plenty of questions at the end.  Please feel free to get in touch if you'd like to know more.

Read More

Southern Delay Repay

While my train was late again this morning, I found the time to look at the Southern Rail passenger charter.  I discovered that when the trains are over 30 minutes late the calculation that Southern make to establish what level of compensation is that they assume I make 546 annual journeys, 10 x 52 plus 1 per weekend per 4 weeks (26).

If the train is 30 minutes late I get 50% of 1/546 of my annual pass.  That equates 1/1092 of the total costs, or less than 0.1% - thanks

In reality with 9 bank holidays (18), 5 weeks holiday (50) and no weekend travel and no sickness, that's more like 546 - 18 - 50 = 478 journeys.  Essentially whichever way you cut it, absolute fleecing...

Read More