treason, as Queen Elizabeth said, is seldom easy to get right.26
At the acme of dynastical insecurity in November 1918, when the monarchies in Austria, Bavaria, Hungary, Prussia, Saxony and Württemberg followed the Romanov empire into extinction, Sir Basil Thomson, the bristling, pushy head of Special Branch at Scotland Yard, wrote a memorandum intended for the eyes of King George V. ‘Every institution of any importance has depended during the war for its existence on an intelligence organization,’ he began with his usual bounding confidence. The Foreign Office, the Admiralty, the War Office and the Ministry of Blockade all had departments collecting data, evaluating rumours, making predictions and trying to stabilize the future. Additional officials in Downing Street were amassing political intelligence for the Prime Minister, Lloyd George. His predecessor, Asquith, had fallen from power in 1916, Thomson continued, ‘not so much because he failed in policy, as because he had no intelligence organization to keep him warned of the intrigues and movements around him’. Similarly, among factors in the recent Russian revolution and overthrow of the Romanov dynasty, defective intelligence had a leading part: ‘Petrograd was in the hands of the revolutionaries before any hint of trouble was heard at Tsarsky’ (Tsarskoye Syelo, the imperial compound outside the capital). Thomson’s lesson for the Royal Household was terse:
(1) The only safe organizations are those that possess an efficient intelligence system.
(2) Those persons or organizations that have failed to develop such systems have been destroyed.
Statecraft had mutated. Europe’s age of intelligence had begun.1
Opposing power blocs had different explanations for these new necessities. Soviet Russia attributed the world’s great changes to the communist revolution of 1917, and to the irresistible impetus towards the dictatorship of the proletariat. The European powers attributed them to the convulsion of continental warfare in 1914–18. Certainly the clashes of the Russian, British, German and Austro-Hungarian empires, and of the French and American republics, had changed their governments’ attitudes to their populations. For centuries monarchs had levied troops to fight wars, governments had repressed civil disorder and reformers had tried to harness popular sentiments. But the military, industrial and transport mobilization of 1914–18 turned the civil population into a new concept called manpower. People of working age – women as well as men – were deployed as a war resource in factories and transport systems as well as on battlefields. The Defence of the Realm Act was enacted in London in 1914, and extended at intervals so as to manage the mass of adults to an unprecedented extent.
When the Russian revolution erupted in 1917, MI5 was focused on German espionage, subversion and sabotage. With the start of the Comintern’s international activities and the foundation of the CPGB in 1920 it changed target to Bolshevism. It relied on the police officers of the Metropolitan Police’s Special Branch, which chewed communists, anarchists, Indian nationalists, pacifists, atheists, self-important but insignificant cranks, Soho rakes, unemployed marchers and mutinous merchant seamen in its greedy, indiscriminate maw. Special Branch reports were often unimaginative if not obtuse. ‘I should wait a long, long time before acting on the advice of the present authorities at Scotland Yard,’ wrote a Tory MP, Sir Cuthbert Headlam, at the time of a botched police raid on the offices of the Soviet trading agency ARCOS in 1927. Special Branch lost control of monitoring domestic subversion after the discovery in 1929 that it had been betrayed by two Bolshevik informants, Hubert van Ginhoven and Charles Jane. Complicated cross-jurisdictional clashes between MI5 and SIS were considered at ministerial level by the Secret Service Committee in 1919, and again by senior officials in 1921, 1922, 1925 and 1927. Committee members found it hard to adjudicate between the two agencies. A resolution was not reached until Kell’s organization was named as the lead national security service in 1931.2
MI5 was primarily an advisory agency, which existed to inform government decisions and to assess and manage risks. Its staff collected, filtered, indexed and filed information gleaned from confidential informants, passport and customs officers, intercepted mail, the garbled chatter heard by covert bugging of offices and telephones, watching of addresses, shadowing of individuals and surveillance of bank accounts, public meetings and publications. Counter-intelligence officers used this data to assess the risks posed by individuals who might be subversives or spies. They resembled historians scouring documentary fragments, unravelling confused memories, checking false trails, re-evaluating doubts and discounting persecution complexes. Their re-examination of past bungles was often more informative than success stories. In the search for long-term patterns, material from multitudinous sources was assembled, allowed to fester, pondered, evaluated, deconstructed, rejected and revised. Every intelligence service was a paper-driven bureaucracy. Officers in MI5, the Cheka, OGPU and the NKVD commissioned reports, compiled profiles, read and reread their dossiers. Every small detail was committed to paper. Successful counter-intelligence usually means following a paper-trail.
The arrest of spies was not the invariable first object of counter-espionage. It was often more profitable to watch and learn. If a spy was allowed to go free, watchers could study his methods and identify his contacts. That was why SIS betrayed its inexperience of domestic counter-espionage when, in 1927, its men arrested Georg Hansen, the Soviet handler of the spy Wilfred Macartney, three days after his arrival in England, before he had found his bearings or met his contacts. Arrests might lead to exciting public denouements, but counter-espionage officers prefer to accumulate and refine intelligence rather than arrest suspects, put them on trial and thus risk disclosure of their methods.
One early example of their watch-and-learn procedure can be given. Theodore Rothstein was ‘a short, stumpy, bearded, bespectacled revolutionary who looked like Karl Marx’. Before 1914 he had collaborated with Wilfrid Scawen Blunt’s campaigns against British imperialism in Egypt and India: during the war he was a triple agent working for Lenin’s Bolsheviks, British military intelligence and the Turkish secret service. In 1919 he wished Leonard Woolf, a former colonial official turned radical activist, to publish Lenin’s recent speeches and insisted on delivering the texts in person. Woolf was instructed to walk on the inside of the pavement along the Strand eastwards towards Fleet Street on a Wednesday afternoon, timing it so that he passed under the clock of the Law Courts at 2.30. At that exact moment he met Rothstein walking westwards from Fleet Street on the outside of the pavement. Rothstein carried in his right hand an envelope containing Lenin’s speeches, which, without either man speaking or looking at each other, he transferred into Woolf’s right hand. These precautions were in vain, for Rothstein was shadowed everywhere by Special Branch and the handover was seen. A few days later the police raided Woolf’s printers and seized the documents. Although Rothstein was kept under surveillance, he had been neither arrested nor dismissed from his wartime job at the War Office: officials judged that it was better to keep him in sight rather than expel him to Russia, where out of reach he might prove a dangerous opponent. This was a pattern of behaviour that was to be repeated in cases over the next century.3
In 1919 there was a sensation when the miners’ leader Robert Smillie, as a member of the Royal Commission on the Coal Industry, browbeat three coal-mining magnates, the Duke of Northumberland, the Marquess of Londonderry and the Earl of Durham, and quoted egalitarian extracts from the Christian gospels at them. ‘The public is amused by the spectacle, but
