In this article I will introduce a methodology for trader performance evaluation that is simple, intuitive and remarkably effective when used in conjunction with the other tools the risk manager has at his disposal, particularly common sense. But first, to motivate and inform the discussion, I'll tell a war story.

War Story

Once upon a time (July 1987) we hired an options trader/market-maker/broker-dealer for one of our overseas markets. At the time, I had not yet completed writing the risk control/risk management portion of my program, and management was only coming to terms with the idea I proposed of a using a probabilistic metric. We operated, back in those Dark Ages, with a system of ex-post-facto cutoffs; in this fellow's case, the stop-loss for his job was $500,000. The first few weeks were uneventful:

But then came the global stock-market crash of 1987:

Now, we had a problem: To sack or not to sack, that was the question. I had worked with this guy for nearly a quarter of a year and had observed his technique, essentially a tightly controlled short-gamma-with-overhedging-directionally technique, and had a qualitative feeling about his ability to run a book and ultimately produce serious money. But I also knew that my qualitative feelings were not sufficient to convince senior management (Note: Senior management operate under quite different rules than normal human beings. One imperative is to not shift heat upwards, to find the lowest plausible level for responsibility for any risk taken that turns out wrong, and make sure that blame sticks at that level) .

So, in a nutshell, the problem became one of coming up with a tool for analysis, a nice, objective-looking, quantitative-feeling means of expressing the qualitative assessment I had already made, on the one hand (it is amazing how many analyses find their genesis in this need to justify a position already taken...) and, on the other hand, to provide a reality check for my own subjective judgement --- was it only my qualitative impression that there was something there?

If there really was something there, as I had hypothesised, it would show up in a reasonable analysis. The first step was to look at the P/L fluctuations (this has become standard practice in the form of Sharpe ratios for investment managers, but the time horizon here was short).

After looking at the problem from many different angles, I ultimately hit on analysing the daily P/L. So, I threw a daily P/L Oscillator accross the cumulative P/L history:

A picture begins to emerge. Later on, we shall see that the daily P/L oscillator shows a particular pattern that is almost prototypical for a short-gamma options trader (But I am getting ahead of myself here). Now, let's look at the daily P/L in isolation, to highlight the patterns we see:

 In analysing this particular trader's performance, we note the following:

1) There is an indication of increased risk in the immediate period leading up to the disaster---it is impossible to know in isolation whether this represents the market fluctuation on an essentially controlled portfolio, or whether, on the other hand, it represents the trader taking on more risk.

2) The pattern of losses and gains is actually quite distinctive, and this is brought into sharper focus through the medium of daily P/L graphing. We can see a steady positive daily P/L for the most part, interrupted by infrequent, larger, losses. This quality is the most salient point of the analysis.

Now in any analysis that you undertake, it is generally welcome (although certainly not absolutely necessary) when the results are in accord with your intuition (particularly if your intuition is reliable and informed). In this case, we know that the subject of the analysis was a short options trader who tended to over-hedge in a controlled way in the direction of his view. The short options technique tends, in capable hands, to produce a plethora of small gains that outweigh the occasional large loss that occurs from time to time. Now the pattern that emerges from the analysis is in accord with our intuition of what this sort of trading will lead to. Equally (or perhaps more) importantly, we are now presented with a graphical prototype of a trading style.

The question remains: Is this guy any good? After all, he did lose $1.1 m in a single day. Phrased differently, was he stupid or just unlucky? The graphical analysis combined with descriptive statistics will point us in the direction of the answer. (Note: In the first place, statistical descriptions here have to be taken with a pinch of slat---we have less than three calendar months, about sixty trading days between the hiring and the catastrophic loss. Moreover, after nearly eleven years of running traders, I can assure you that, out here in the field, the daily standard deviation will always be greater than the mean daily P/L, although you may start to see "significance" in results of weekly or greater periodicity. More on this in another Scribbling). The descriptive statistics for the period are as follows:

Are we any smarter now? Well, not really. For this trader's results to be positive, it is necessary that the gains outweigh the losses. This does not mean that the size of gains be greater than the size of losses, or that the frequency of gains be greater than the frequency of losses. It means simply that the combination of frequency and size must be greater for gains than for losses. Overall, if we look again back to the second graph above (showing both cumulative and daily P/Ls) , we will see this borne out by the slope of the cumulative P/L graph overlaid on the daily fluctuations graph. We have this in nascent form on the graph.

Moreover, as we look at the relative magnitude of losses vs. gains, we find that it manifests an element of good, tight, control. Combining this with the knowledge that --- in this market --- 20 October 1987 was a very unusual day (to say the least) and the fact that the graphics told a story that was consistent with my prior qualitative knowledge and intuition, I came to the conclusion that what he had on our hands was a good, tightly-wound short options trader.

With the help of the graphical returns analysis, I was able to prevail upon management to override the ex-post stop-out figure, and keep the guy on.

Epilogue

So, what was the upshot of it all? First, let's look at the continuation of the trader's track record:

You will note that, although as time goes on, we see the reduction in dispersion, the same basic pattern remains: A large number of small daily gains, punctuated every now and again by a largish loss. As I grew more adept at reading these particular tea leaves, it began to dawn on me that what we have is a trading "signature". The analogy to a human signature is not a bad one: Each time an individual signs his name, the signature looks more or less the same, yet no two signatures are identical. A pattern is recognisable to all, and uniquely identifies the individual who signs it. So too with an experienced and developed trader. The pattern of daily P/L results taken together with the cumulative P/L form a unique trading signature for the disciplined individual. The case above (and shown completely in the last graph below) is an almost prototypical short gamma trading signature. (Note: the marked reduction in fluctuations following April 1988 coincides with the installation of our then-new risk management system, modestly called the "Halperin Portfolio Management System").

Well enough of the analysing and scrutiny of patterns. Did I make the correct call in convincing management to keep our trader on? After all, money counts in this game (at least for most of us). Here, then, is the follow-on P/L picture:

As you can see, the trader went on to make $10mm for us over the next two years, nearly always following his short-gamma pattern. The scale of the post-hoc cumulative profits totally dwarfs the post-hoc risk. And the 1.0mm hiccup that motivated the development of the signature analysis tool looks rather overpowered by the profits that followed. Viewed in this light, the trading signature analysis turned out to be a handy little addition to the Risk Manager's toolbag.