Forem: Robert Morschel

Is your code S.A.F.E?

Robert Morschel — Thu, 05 Apr 2018 13:36:07 +0000

The website Information Is Beautiful has an illuminating chart for the World's Biggest Data Breaches. Not only does it show that breaches have become more common in recent years, but also that many of these breaches were insider jobs. Hacking has become an industry in its own right and, if you're a software developer, security should be your concern. This is particularly true in the world of distributed software where your environment is often not entirely under your control and should not be blindly trusted. You need to ask yourself: is my code is S.A.F.E?

S : Sensitive

Sensitive code looks after its sensitive data (passwords, application secrets, client personal detail, session tokens, ...) This means:

Not storing application secrets in version control unless there is tight access-control over the repository. Ideally, move secrets to secure servers, and even more ideally, move secrets into a password manager like HashiCorp's Vault.
Encrypting sensitive data in transit and at rest.
Not logging sensitive data, particularly if you have a log aggregator that makes this data visible to potentially unauthorised individuals within the company.

A : Access-controlled

Whenever your sensitive code is remotely accessed, you need to authenticate and authorise the request, i.e.

Who is the caller?
Are they allowed to do this?
Are they allowed to do this on this particular data?

You need to log requests for auditing purposes, in particular, authentication or authorisation failures.

In addition, consider zoning your network so that not everybody can reach your sensitive services.

F : Foolproof

Do you validate your requests?

It's good defensive programming practice to validate your inputs, but this is particularly true in the world of JavaScript and SQL injection attacks.

Prefer strong-types to strings. If you must use strings, prefer whitelisting to blacklisting checks (today's evil character set might not be tomorrow's).

Fail securely.

When returning errors, don't reveal details of your system's inner workings that a hacker could use (e.g. stack traces or overly specific error messages like "password incorrect" instead of "login failed").

E : Enforced

You don't know your code is secure unless you've checked that it is. This means:

Threat-modeling your design.
Adding security checks to your code-reviews.
Automating your security checks via tests.

Be SAFE. It's dangerous out there.

P.S. If you want to learn more about application security best-practices, check out the OWASP website.

Just Enough Documentation

Robert Morschel — Thu, 23 Nov 2017 14:42:16 +0000

Code hardly every adequately documents itself. Nobody in their right mind wants to learn a complex new system architecture by being presented with an IDE and a link to github. Good documentation is an essential part of any maintainable system.

Good documentation often starts as scribblings on a whiteboard, but if an artefact is useful, it is worth being captured more permanently (even if that's just a photo on a wiki).

Speaking of wikis... just because it's easy to create a page, doesn't mean you should!

Good documentation helps to communicate important ideas.

Good documentation remembers what we tend to forget, particularly reasons for decisions.

Good documentation aims to be just in time and just enough.

Good documentation is useful, which means it has to be accurate and relevant.

Good documentation is owned and looked after. If you remember nothing else of this rant, remember this.

Good documentation is not redundant.

Good documentation knows when it is none of these, and should be archived for the historians.

How To Write Good

Robert Morschel — Wed, 17 May 2017 14:13:16 +0000

I edit IG's technical blog. Here is a list of guidelines for writing articles that I've shared with my developing writers.

Can you describe the point of your article in one interesting sentence?
Identify the reader. Who are you writing for? Why do they care?
Start well. Get their attention.
Be interesting and enthusiastic.
Don't be too formal or too chatty.
Be specific - details are often more interesting than abstract concepts.
Prefer simpler words.
Omit needless words. It is very important to basically avoid fluff words because they are rather empty and sometimes a little distracting. Vigorous writing is concise.
Don't ramble. Don't repeat yourself, even if you really value your opinion. Did I mention the rambling? Less is more.
Prefer shorter sentences and paragraphs.
Use the active voice.
Be easy to read. Flow. Tell a story.
Read your writing out loud.
End well. Summarise and conclude.
Edit ruthlessly. Take a break. Repeat.
Spell Czech.
Read the Strunk's "Elements of Style", often.

I've worked with writers of varying abilities, and have found that the biggest obstacle is not lack of talent, but of self-belief.

You can write. Trust me.

IG Public API - Lessons Learned

Robert Morschel — Mon, 15 May 2017 08:27:34 +0000

A few of us built the IG public API in around 6 months, a feat we were all tremendously proud of.

But mistakes were made, lessons were learned, but unfortunately too late.

Here is what I wish we'd done differently.

To Gateway Or Not?

We evaluated a number of API gateway products (Apigee, Layer7 and WS02), but decided to build our own. The reasoning was:

Our trading API was latency sensitive. The fastest of the products (WSO2) added 3ms, which may not sound like much, but in trading, even online trading, it is.
Our backend APIs were already REST, so all the fancy ESB-type integration functionality the gateways offered were of no interest to us.
We had security solved, so didn't need OAuth (and anyway, latency went up to around 50ms with OAuth on!)
The gateway web portals were too prescriptive for our liking - we wanted ours to be heavily IG-branded and customised, not just "white-labelled".

So we built a thin API adapter framework on top of our existing internal REST APIs, doing little more than adding basic API key security, transformation and documentation generation from source.

It turns out our security solution was inadequate, and we did need OAuth. We ended up using OpenAM (which is an excellent freemium open source security platform, by the way).

We invested too much time in our own web portal, and still, in my view have ended up with a sub-standard offering. Better to have been happy with 80% of a 3rd party offering, and ended up with a more polished overall product.

We used IG's RESTdoclet at the time to generate our API documentation, but should have used Swagger (which we subsequently moved to internally). A number of the gateways supported Swagger.

Having said that, think carefully before you generate documentation. If you put your documentation in source code, make sure that it is clearly marked as content that will be visible on the web, and subject it to rigorous review by a skilled content writer. Don't assume developers know how to write content. They usually don't.

We're now considering moving the adapter layer into a separate, how shall we say... er ... gatew. No, I can't say it.

Not so RESTful

I like REST. It offers a very simply API model which is easy to get to grips with for the average punter. We did a lot of research before making the following mistakes:

Modeling using URI

We tried to get too clever here, and model a URI hierarchy that mirrored our domain resource taxonomy. Unfortunately the model turned out to be too complicated for a simple resource tree, and we ended up with an ugly set of URIs in some instances.

Using HTTP DELETE

In finance systems, there is often very little opportunity to really delete things - at best one marks things as deleted. So that is more of an update (PUT) than DELETE. But we persisted, and found two cases where we could use DELETE:

logout: DELETE /session (the argument being that logging in is just creating (POST) a session, and logging out is deleting the session.) Intuitive as a brick.
closing a position: the problem here is that positions can also be partially closed. So we thought, no problem, just put that data in the request body. This is bad news and not widely supported, so we ended up using PUT (as we should have done).

We still have DELETE /session to logout, but nobody uses it. This hurts.

Using HTTP status codes

It seemed brilliantly simple at the time. 200 means success. 400 error range means the client needs to fix something. 500 error range means the server has to fix something.

Many objections to this will be religious, some suggesting that HTTP status codes were never intended for APIs, and only 200 should be used. OK, fine, but then you have to handle 401 and 403 security errors, so why not use 404 (not found) for lookup errors and 400 (bad request) for validation errors? And 500 range for system errors?

This all falls apart when you try to return error details in the response body. For example our CDN (Akamai) happily replaced our custom 500 error message with its own.

So in the end we stopped using 500 message response bodies, which is a shame. A good pattern here is to generate an internal GUID for a logged system error and then return that to the caller, so they have something to quote when they phone up support.

Using headers to version

We didn't want to clutter our beautiful URIs with versions, so we put the API version in the header, and applied it at the "operation" level, i.e. URI + HTTP method.

I could talk at length about versioning, but will save that for a future post. In summary I think API versioning should be as follows:

Version number at the beginning of the URI.

non breaking change (no version number change)
breaking change (version number change)

To B2C or B2B?

Being "agile" means we sometimes forget to think things through completely before starting to cut code. We didn't think through who would be using our API. We thought: REST is easy, so our customers will finally be able to automate their trading. They will just come, and trade... kerching!

And many did, but many struggled, and many didn't even trade: they just got our free (i.e. no kerching!) market prices. But we had a lot of support calls, ranging from bugs (which is cool) to "Can you tell me how to write a program?" (which is understandable but irritating. I was so tempted to send a link to my university degree course).

In one sense I don't regret opening up the API to the masses, since this really put our API through its paces, but the power-users of the API ended up building applications (with the help of developers they hired) - which is precisely our B2B model, and where the real money is, in my humble opinion.

But we spend so much time supporting the punters, and it's just not cost-effective.

We made the mistake of "building it and hoping they would come", without thinking who would come, and who should come.

So you want a public API?

Robert Morschel — Sat, 13 May 2017 20:59:59 +0000

(Originally posted on: labs.ig.com)

So you'd like your customers to integrate with you, and what better way to do this than a shiny new public web API? Perhaps you have a suite of existing web services for your web and mobile platforms, so it's simply a matter of "exposing" these, right?

But is your existing API good enough to expose as is? What exactly makes for a good public API?

Does the API meet your customers' needs?

This may seem an obvious point, but very often public APIs, particularly if derived from existing APIs, expose what the company does, not what the customer needs. An example of this is the IG EPIC instrument identifier. Used internally to identify instruments (markets), these are proprietary to IG and difficult for customers to relate to real-world identifiers (RIC, SEDOL and the like).

Does the API meet your business goals?

"Build it and they will come" is certainly one approach to justifying building an API, but what if the wrong people come, the time-wasters, those who will consume all your support and development bandwidth without offering any real value to your business - often at the expense of the serious customers who will bring real revenue? Does it even make more sense to offer your API to individuals, as opposed to institutions?

The cost model of the API needs to be defined. Is it free or restricted? Should there be price bands? Should cost be related to type of usage? How will you enforce this? Having a clear and understanding of the API business model (supported by detailed API usage metrics) is critical to the success of the venture.

Is it consistent?

A consistent API looks and behaves in a predictable manner. This involves deciding on a fundamental access protocol, and layering above that standard conventions: a functional model (consistent URI scheme, resource types, attribute names, types, …), and a non-functional model (versioning, error handling, security, retry handling …)

Is it available?

An available API has well-defined and monitored SLAs, is resilient to failure, scalable, versioned, repeatedly tested so that API interface or downstream changes do not impact customers, secure and audited (both for the API's sake, but also for the customer's.)

Is it usable?

A usable API is thoroughly, accurately and consistently documented, both in terms of an API reference, but also in terms of Getting Started and User Guides to introduce clients to non-trivial aspects of using the API. This is where API consistency helps immensely.

Consider implementing a portal for customers to easily access API documentation, as well as to try out APIs interactively. Offer support contact details and a forum to allow customers to interact with your company, but also to build up a community offering mutual support. Decide what your internal customer support team structures will look like. Consider setting up a customer-facing technical team with ready access to a team of internal API developers.

Good public APIs do not just magically appear overnight - they are complex and costly to build well, and to run.

Are you sure you want one?

Illusory Comforts

Robert Morschel — Sat, 13 May 2017 08:15:51 +0000

I like process.

I appreciate that this is probably because it's in my nature to like order and predictability, but good process adds value, can often be automated, and frees up developers to do what they do best: create excellent software.

Bad process on the other hand is a killer.

It's a killer because it slows everything down.

It's a killer because it doesn't work, and is often bureaucratic.

But mostly it's a killer because it gives those in charge the illusion of progress.

I worked with a place once that outsourced their software development capability to an outfit that had CMM level 5 accreditation, and could prove it ... but developed the worst software I had ever encountered. Yet senior management were blissfully unaware of the fact, or perhaps they were and didn't care. They had achieved their outsourcing objectives, reduced resource costs, and achieved CMM level 5 in the process! I bet bonuses were good that year.

A bad leader does not trust his team to do their job. He tries to measure them, to enforce success via metrics and bad process. It offers him an illusory comfort of progress to see the hours worked and lines of code per day metric increasing; and damn those inconvenient holidays!

A good leader understands that his team consists of people, not resources. Humans who need autonomy, mastery and purpose. Motivated individuals who will take pride in their work, and do their best to get the job done well.

And that's all you can ask of them, really.

Evil Session Tokens

Robert Morschel — Tue, 09 May 2017 21:18:17 +0000

So you've built a new web application, SIMPLES.COM. Clients can login to the secure site, over HTTPS of course, and a session token is issued. Since we don't want the client to have to log in every time, we give the token a long lifetime, or at least a way of using it to get a new one.

The application persists the token on the browser using cookies. You did consider using browser local storage, but cookies seemed the best way to guarantee wide browser compatibility.

The application then validates every HTTP request, using the session token in the cookie, and authorises access to the client as appropriate.

Sorted.

Enter the hacker.

Client Joe Bloggs receives an email with a phishing link to dodgy website S1MPLES.COM (note the name is different), and clicks on it. Joe is taken to a page where a few sneaky GET requests are sent to the real SIMPLES.COM website, and Joe's browser helpfully supplies the session cookie, since it's the correct domain, and the hacker now potentially has control of Joe's account via a number of Cross-Site Scripting Attack attack vectors. (Edited previously incorrect statement about full access, which isn't applicable to this scenario - see comments below)

How do we fix this?

The first thing to do is not use cookies for authentication. The web application must attach a session token header to each request. That way the above Cross-site Request Forgery attack is not possible.

However, you now have a long-lived session token being passed around, but that's OK, because you use HTTPS.

Except you don't. Not all the time.

SIMPLES.COM is accessible over HTTP. Only the login page and secure site are via HTTPS.

So Joe, being a lover of coffee, and free WiFi, gets caught in a man-in-the-middle attack by Mr Evil with a cheap, portable WiFi router. Mr Evil intercepts the requests, and of course the session session token sent with each request.

Mr Evil now has full access to Joe's account, for a long time.

The answer is to make SIMPLES.COM full HTTPS, and to use HSTS to ensure no opportunity for man-in-the-middle exists. And to add all the recommended security headers, e.g. content security policy.

Cool. Mr Evil shrugs, and decides to pick on someone who hasn't read this article.

Much later, Mr Bored Developer is browsing through the application logs (helpfully made available via a log aggregator), when he notices that the session tokens are logged for all to see.

Not so cool. Developers are not immune to the lure of crime, even when writing Clojure.

So what do we do?

One way is to use an OAuth-style approach to issue, not one session token, but two: a short-lived access token, and a long-lived refresh token. The refresh token is stored in the application browser, but is never used for access, only to request new access tokens.

Of course these access tokens could still be leaked, but being short-lived, will expire very quickly.

And there you have it.

Are Your Children Safe?

Robert Morschel — Tue, 09 May 2017 20:59:45 +0000

Your company has value. It has customers. It has reputation. It has money.

All these are of interest to criminals: whether a lone hacker, a sophisticated organisation, or perhaps a disgruntled, debt-ridden or blackmailed employee.

So I ask you: Is your software secure? What is the risk of a breach? Are your children safe?

If you don't know the answer to these questions, then you are a hack waiting to happen. It's like a life insurance policy. You don't have to have one, but can you afford the consequences of not having one?

You need to think about security, and this applies equally to externally facing systems as to internal ones.

I would like to suggest that every single application or service you write needs a threat model. This is a very simple OWASP tool for assessing components in terms of STRIDE threats (Spoofing, Tampering, Repudiation, Information Disclosure, Denial of Service, Elevation of Privilege).

I would furthermore like to suggest that every threat you find needs an associated set of tests to be written.

You do have security tests, right? Right?

Essential further reading: OWASP.

OAuth for Dummies

Robert Morschel — Tue, 09 May 2017 16:10:24 +0000

If you've ever accessed a website using your Facebook or Google credentials, then you've encountered OAuth. OAuth is a authorisation web protocol for granting websites access to some of your personal information or access rights to some system. The analogy usually given is that of a valet key, which makes no sense to me, as I own a rusty VW Beetle, but let's not worry about that.

If you're building such a website, you'll want to register it with the OAuth provider (Google or whatever). They will give you an application id and secret. This secret allows you to speak OAuth to the provider.

So, much later, Joe Bloggs comes along and wants to access your website. Your website sends a request to the OAuth provider (with your application id and secret) and gets given a conversation token.

Then, you redirect the user to the OAuth provider (but include the conversation token and a callback URL, which will have been included when you registered your app). The OAuth checks the conversation token and URL, and then works out who the user is (e.g. asking you to log in) and then presents a consent screen: "Do you want to allow application X to access your Y?" You check out the Y list and say OK (or not). The OAuth provider then sends an HTTP post via the browser back to your website, with two tokens:

an access token (the valet key), which allows you to ask the OAuth provider for Y things
a refresh token (for when the access token expires, which will be frequently)

Hang on a minute! What about these Y things? How does the OAuth provider know what the website wants rights for?

OAuth calls that "scopes". A scope is a name of a user "thing" that you're allowed access to. It could be a user name, email, or a set of access rights. When the website does OAuth, it adds a some scopes to the OAuth request, and if they are in the permitted list, they get presented to the user.

And that is basically it. There are some other nuances and stuff, but my car's broken down again and I need to run.

Self-documenting Architecture

Robert Morschel — Tue, 09 May 2017 08:03:00 +0000

For a software architect, and someone who tinkers with words, this is perhaps an odd thing to say, but it is true: I hate documentation.

I hate it because it is laborious, very often serves no purpose other than to have been produced, and rapidly gets out of date. I have followed methodologies where every UML model under the sun is produced, code is generated, and round-trip engineering is attempted. It never really worked.

But documentation can add value. Code is a very low-level, cumbersome way to get to grips with a system, even if it is allegedly "self-documenting". A few high-level component or sequence diagrams can add a lot of value. Documentation helps you to navigate a system, and understand why something was built.

So what to do?

I think the first thing to do is to distinguish between transient and permanent documentation.

Transient documentation is correct at a point in time, and very often only exists on paper or a whiteboard. It serves to help discuss a solution, but has a very short life span. After it has served a purpose, it usually gets to rest in peace in some archive (or dustbin).

Permanent documentation, however, continues to exist alongside the system, a living description that helps newcomers to understand what was built, and more importantly, why. But permanent documentation needs to be owned and managed. And being managed, needs to be kept up to date. This means that if a project impacts the system, the owner(s) of the permanent documentation of that system need to be notified, so that the documentation can be updated. Of course this is costly, so you do it only to documents you care about.

But this is not the end of the story. Generating navigable documentation from your systems is a powerful way of supplementing your useful formal documentation.

For example:

using automated API documentation tools like Swagger to generate callable REST service interface documentation.
using deployment scripts to publish deployment information to a configuration management database.
using application performance monitoring technology like AppDynamics to show system deployments and interactions.
using standardised logging and log aggregation (operational intelligence) tools like Splunk to help trace message flows through systems.

Some of these tools cost money, but there are usually decent open source options available, and anyway, if they save you using MS Word, it will be worth every penny.

P.S. I love wikis - they are great for collaborating, but their ease of use means that they are most often cluttered with half-baked attempts at documentation that, at best, offer no value, at worst, offer misinformation. Permanent documentation needs better controls and rigour than a wiki encourages.

The Importance of Simplicity

Robert Morschel — Tue, 09 May 2017 07:27:09 +0000

I remember way back when, a number of colleagues being engrossed in the latest Obfuscated C++ challenge, an exercise in making code so concise, so cryptic, that even the hardened coder paled.

This always struck me as a very odd thing to want to do. Why make things more complicated than they have to be? Surely code simplicity and readability are the most important thing? After all, software only spends a small fraction of its life being created. The rest of the time it is being maintained, usually by different people who are probably not going to appreciate your obscure brilliance.

Software is much like conversation. It's not just what you say that's important, but how it's heard and understood.

Should architects code?

Robert Morschel — Mon, 08 May 2017 19:16:51 +0000

There is this rumour going around that good architects need to write (production) code. I think the term was coined by Scott Ambler of agilemodeling.com, or was it ThoughtWorks? Anyway, I don't care, because I disagree.

See, I did write code, a lot of it. I was good at it, and I still like to write code for my own pleasure, but my current, full-time role as an enterprise architect does not involve much coding. As a result, I have become slightly detached from the way code is written in our company (tools, processes, etc.) and when I do, it is laborious and frustrating. In fact, I would respectfully suggest that letting me near production code might: a) not be the best use of my time, b) be dangerous.

It's not because I'm older, or wiser, or slower, or too important to code. It's because I spend my time thinking about different abstractions: higher-level ones. The principles are the same, but the moving parts are larger.

See, I write enterprise code.

My IDE is Powerpoint ... or Word, if I'm feeling adventurous.

I am like the town planner. Sure, I could have a go at building a house, but would you really want to live in it?

I know I wouldn't.

But what do you think?