On Friday, August 17 2007, 21 of Wall Street’s most influential investors met for lunch at George Soros’s Southampton estate on the eastern end of Long Island. The first tremors of what would become the global credit crunch had rippled out a week or so earlier, when the French bank BNP Paribas froze withdrawals from three of its funds, and in response, central bankers made a huge injection of liquidity into the money markets in an effort to keep the world’s banks lending to one another.

Although it was a sultry summer Friday, as the group dined on striped bass, fruit salad and cookies, the tone was serious and rather formal. Soros’s guests included Julian Robertson, founder of the Tiger Management hedge fund; Donald Marron, the former chief executive of PaineWebber and now boss of Lightyear Capital; James Chanos, president of Kynikos Associates, a hedge fund that specialised in shorting stocks; and Byron Wien, chief investment strategist at Pequot Capital and the convener of the annual gathering – known to its participants as the Benchmark Lunch.

The discussion focused on a single question: was a recession looming? We all know the answer today, but the consensus that overcast afternoon was different. In a memo written after the lunch, Wien, a longtime friend of Soros’s, wrote: “The conclusion was that we were probably in an economic slowdown and a correction in the market, but we were not about to begin a recession or a bear market.” Only two men dissented. One of those was Soros, who finished the meal convinced that the global financial crisis he had been predicting – prematurely – for years had finally begun.

His conclusion had immediate consequences. Six years earlier, following the departure of Stan Druckenmiller from Quantum Funds, Soros’s hedge fund, Soros converted the operation into a “less aggressively managed vehicle” and renamed it an “endowment fund”, which farmed most of its money out to external managers. Now Soros realised he had to get back into the game. “I did not want to see my accumulated wealth be severely impaired,” he said, during a two-hour conversation this winter in the conference room of his midtown Manhattan offices. “So I came back and set up a macro-account within which I counterbalanced what I thought was the exposure of the firm.”

Soros complained that his years of less active involvement at Quantum meant he didn’t have the kind of “detailed knowledge of particular companies I used to have, so I’m not in a position to pick stocks”. Moreover, “even many of the macro instruments that have been recently invented were unfamiliar to me”. Even so, Quantum achieved a 32 per cent return in 2007, making the then 77-year-old the second-highest paid hedge fund manager in the world, according to Institutional Investor’s Alpha magazine. He ended 2008, a year that saw global destruction of wealth on the most colossal scale since the second world war, with two out of three hedge funds losing money, up almost 10 per cent.

Soros’s main goal was to preserve his fortune. But, as has been the case throughout his career, his timing and financial acumen enhanced his credibility as a thinker, and never more so than in 2008. In May and June, after more than two decades of writing, he hit bestseller lists in the US and in the UK with his ninth book, The New Paradigm for Financial Markets. In October, he received an invitation to testify before Congress about the financial crisis. In November, Barack Obama, whom he had long backed for the presidency, defeated John McCain.

“In the twilight of his life, he’s achieved the recognition he has always wanted,” Wien said. “Everything is going for him. He’s healthy, his candidate won, his business is on a solid footing.”

. . .

Many comparisons have been drawn between 2008 and earlier periods of turmoil, but the historical moment with most personal resonance for Soros is not one of the conventional choices. The parallel he sees is with 1944, when, as a 13-year-old Jewish boy in Nazi-occupied Budapest, he eluded the Holocaust.

Soros credits his beloved father, Tivadar, with teaching him how to respond to “far from equilibrium situations”. Captured by the Russians in the first world war, Tivadar was imprisoned in Siberia. He engineered his own escape and return home through a Russia convulsed by the Bolshevik revolution. That sojourn stripped him of his youthful ambition and left him wanting “nothing more from life than to enjoy it”. Yet on March 19 1944, the day the Germans occupied Hungary, the 50-year-old sprang into action, rescuing his immediate family and many others by arranging false identities for them.

Before the invasion, George was still enough of a child, his father thought, to need a bit of parental coddling. Yet the teenager who spent the war living apart from his parents under a false name found the danger exhilarating. “It was high adventure,” Soros wrote, “like living through Raiders of the Lost Ark.” And as the latest financial crisis gathered momentum, he admitted to the same thrill. “I think the same thing applies again. I feel the same kind of stimulation as I felt then,” he told me.

Part of the stimulation is intellectual. Soros’s experiences in 1944 laid the groundwork for the conceptual framework he would spend the rest of his life elaborating and which, he believes, has found its validation in the events of 2008. His core idea is “reflexivity”, which he defines as a “two-way feedback loop, between the participants’ views and the actual state of affairs. People base their decisions not on the actual situation that confronts them, but on their perception or interpretation of the situation. Their decisions make an impact on the situation and changes in the situation are liable to change their perceptions.”

It is, at its root, a case for frequent re-examination of one’s assumptions about the world and for a readiness to spot and exploit moments of cataclysmic change – those times when our perceptions of events and events themselves are likely to interact most fiercely. It is also at odds with the rational expectations economic school, which has been the prevailing orthodoxy in recent decades. That approach assumed that economic players – from people buying homes to bankers buying subprime mortgages for their portfolios – were rational actors making, in aggregate, the best choices for themselves and that free markets were effective mechanisms for balancing supply and demand, setting prices correctly and tending towards equilibrium.

The rational expectations theory has taken a beating over the past 18 months: its intellectual nadir was probably October 23 2008, when Alan Greenspan, the former Federal Reserve chairman, admitted to Congress that there was “a flaw in the model”. Soros argues that the “market fundamentalism” of Greenspan and his ilk, especially their assumption that “financial markets are self-correcting”, was an important cause of the current crisis. It befuddled policy-makers and was the intellectual basis for the “various synthetic instruments and valuation models” which contributed mightily to the crash.

By contrast, Soros sees the current crisis as a real-life illustration of reflexivity. Markets did not reflect an objective “truth”. Rather, the beliefs of market participants – that house prices would always rise, that an arcane financial instrument based on a subprime mortgage really could merit a triple-A rating – created a new reality. Ultimately, that “super-bubble” was unsustainable, hence the credit crunch of 2007 and the recession and financial crisis of 2008 and beyond.

As an investor and as a thinker, Soros has always thrived in times of upheaval. But he has also remained something of an outsider. He recalls how he “discovered loneliness” when he arrived to study at the London School of Economics in 1947. Later on, as he worked his way up from being a journeyman arbitrage trader in London and then New York, to running one of the world’s most successful hedge funds, Soros remained, in the words of one private equity acquaintance, a bit of “an oddball”, both on Wall Street and in the academic world. He is frequently described as “charming”, yet few see the fit, tanned, twice-divorced billionaire as an emotional confidant. “If I had an idea about India-Pakistan, I would talk to him about it,” Wien said. “If I were having a problem in my marriage, I don’t think I would go and talk to George about it.”

Strobe Talbott, now the president of the Brookings Institute and a former deputy secretary of state, said: “He likes to think of himself as an outsider who can come in from time to time, including to the Oval Office, where I took him on a couple of occasions. But simply hobnobbing with the powerful isn’t important.”

That lack of clubbiness, and the associated trait of iconoclasm, may explain why, for all his worldly success, Soros has had a rather mixed public reputation. His speculative plays, which have often targeted currencies, have earned him the wrath of political leaders around the world. The ambitious, global reach of his richly funded Open Society foundation has prompted some critics to accuse him of suffering from a Messiah complex. He was so effectively demonised by the US right earlier this decade that he kept fairly quiet about his support of Obama, lest the association hurt his candidate. Probably most painfully, his forays into economics and philosophy often have met with considerable scepticism, especially from academia.

The one time and place where he instantly became a highly regarded insider was in the former Soviet Union and its satellites, at the moment the Berlin Wall came down. More completely and more swiftly than any other foreigner, Soros grasped and embraced the systemic transformation that was unfolding, and was rewarded with influence and respect. The question for Soros today is whether, as the west undergoes its own once-in-a-century systemic shock, this arch-outsider will finally find himself in the mainstream in the society which has been his main home for more than half a century.

. . .

Soros’s most famous – or infamous – speculative play as an investor was his bet against sterling in 1992, a wager which won him more than $1bn and earned him the epithet from the British press of “the man who broke the Bank of England”. That bet also turns out to be a perfect illustration of the specific talent which his past and present fund managers agree has been central to his investing success.

Soros’s best-known investment was not, in actual fact, his own idea. According to both Soros and Druckenmiller, who was managing Quantum at the time, it was Druckenmiller who came up with the plan to short the pound. But when Druckenmiller went through his rationale with Soros, in one of their twice- or thrice-daily conversations, Soros told his protégé to be bolder: “I said, ‘Go for the jugular!’.” Druckenmiller duly raised their stake – Quantum and several related funds wagered nearly $10bn, according to interviews Soros gave afterwards – and Soros earned both a fortune and an international reputation.

Druckenmiller, who spent 12 years at Quantum, says that conversation exemplifies Soros’s singular financial gift: “He’s extremely good at using the balance sheet – probably the best ever. He is able to use leverage when he likes it, but he is also able to walk away. He has no emotional attachment to a position. I think that is an unusual characteristic in our industry.”

Chanos agrees: “One thing that I’ve both wrestled with and admired, that [Soros] conquered many years ago, is the ability to go from long to short, the ability to turn on a dime when confronted with the evidence. Emotionally, that is really hard.”

Soros denies any great degree of emotional self-control. “That’s not true, that’s not true,” he told me, shaking his head and smiling. “I am very emotional. I am as moody as the market, so I’m basically a manic depressive personality.” (His market-linked moodiness extends to psychosomatic ailments, especially backaches, which he treats as valuable investment tips.)

Instead, Soros attributes his effectiveness as an investor to his philosophical views about the contingent nature of human knowledge: “I think that my conceptual framework, which basically emphasises the importance of misconceptions, makes me extremely critical of my own decisions … I know that I am bound to be wrong, and therefore am more likely to correct my own mistakes.”

Soros’s radar for revolution is the second key to his investing style. He looks for “game-changing moments, not incremental ones”, according to Sebastian Mallaby, the Washington Post columnist and author who is writing a history of hedge funds. As examples, Mallaby cites Quantum’s shorting of the pound and Soros’s 1985 “Plaza Accord” bet that the dollar would fall against the yen – his two most famous currency trades – as well as a lesser-known 1973 bet that, as a consequence of the Arab-Israeli war, defence stocks would soar. “It’s not that reflexivity tells you what to do, but it tells you to be on the look-out for turn-around situations,” Mallaby said. “It’s an attitude of mind.”

Some Soros-watchers intimate that his vast network of international contacts might be an important source of his market prescience. But it was in the one part of the world where Soros really did have an inside track – the former Soviet bloc – that he made his most disastrous deal. In Russia, as in much of the former Soviet Union, he was intensely engaged with the country’s political and economic transformation. In June 1997, as the Kremlin struggled to pay overdue wages, Soros extended a bridge loan to the Russian government, acting as a one-man International Monetary Fund.

He came to believe in Russia’s commitment to reforms, and to see himself as an insider – two convictions that were his financial undoing. He invested $980m with a consortium of oligarchs who acquired a 25 per cent stake in Svyazinvest, the national telecoms company, deciding to participate because “I thought that this is the transition from robber capitalism to legitimate capitalism”. But instead, the Svyazinvest privatisation turned out to be the moment when the oligarchs redirected their energies from fleecing the state to fleecing one another. Soros, as an outsider, was an obvious casualty. “Never have I been screwed so much since Russia. For them, they get a satisfaction out of doing it.

“It was the biggest mistake of my investment career. I was deceived by my own hope.” In his most recent book he dismisses Russia with a single sentence, further diminished by parenthesis: “(I don’t discuss Russia, because I don’t want to invest there.)”

. . .

On a chilly Monday night in December, Soros took the hour-long drive from Manhattan to the Bruce Museum in Greenwich, Connecticut. He was due to speak at a benefit for the Scholar Rescue Fund, a programme he has partly financed and which, since 2002, has provided safe havens for 266 persecuted academics from 40 countries. After his talk (on the global financial crisis, of course), Soros filed out of the auditorium chatting with Stanley Bergman, a founding partner of the law firm that had sponsored the evening.

“You like the game?” Soros asked his host with a smile.

“Yes,” the white-haired Bergman replied.

Then, in a flash of the competitive spirit that makes Soros an avid skier and player of tennis and chess, Soros asked: “And how old are you?”

“75.”

“I’m 78,” Soros replied. “But what’s the use of good health if it doesn’t buy you money?” The vigorous septuagenarians flashed each other a complicit smile.

According to Wien, Soros likes the game, too: “George loves to be able to show from time to time that he can do it.” But while he loves to play, he is disdainful of a life lived purely to accumulate more chips. His epiphany came in 1981, when he had to scramble to raise money to pay for an investment in bonds. “I thought I would have a heart attack,” he told me. “And then I realised that to die just for the sake of getting rich, I would be a loser.”

For Soros, the solution was philanthropy. “To do something really that would make a significant difference to the world, that would be worth dying for,” he said. “The Foundation enabled me to get out of myself and to somehow be concerned with other people than myself.” Soros’s fortune has given his causes enormous firepower: according to Aryeh Neier, the human rights activist who has been running the Open Society Foundation since 1993, its budget was $550m in 2008 and will increase to $600m this year. By his own calculation, Soros has donated a total of more than $5bn to his causes, primarily directing his giving through his foundation.

“No philanthropist in the second half of the 20th century has done better in deploying resources strategically to change the world,” Larry Summers, the newly appointed head of Barack Obama’s National Economic Council, told me in a conversation early last autumn. Talbott compares Soros’s impact to that of a sovereign nation. In the 1990s, says Talbott, “when I got word that George Soros wanted to talk, I would drop everything and treat him pretty much like a visiting head of state. He was literally putting more money into some of the former colonies of the former Soviet empire than the US government, so that merited treating him as someone with a very high impact.”

Soros’s philanthropic lieutenants report an approach remarkably similar to the investing style observed by his fund managers: he knows how to make big, original bets, and he isn’t afraid to cut his losses when a project isn’t working out. Anders Aslund, an economist who has studied Russia and Ukraine and who has worked with Soros on various projects, believes his philanthropic style “is very much formed by the money markets, which are always changing. He assumes any idea he has now will be wrong in a few years. He is always asking himself, when he has a wonderful project going, ‘When should I stop this project?’.”

Soros’s war chest, and his determination to deploy it beyond the usual blue-chip charities of hospitals, universities, museums or even poverty in Africa, had long made him an occasionally controversial figure outside the US. He was among the western culprits accused by the Kremlin of inciting Ukraine’s 2004 Orange Revolution; his foundation’s offices have been raided in Russia and he was forced to close them down in authoritarian Uzbekistan.

America, it turns out, can also be sensitive to plutocrats using their wealth to address socially contentious subjects. In recent years, his foundation became more active in the US, taking on issues including drug policy. His engagement became more intense during the George W. Bush presidency, when Soros decided that the open society he had worked to foster in repressive regimes abroad was imperilled in his adopted home.

Some admired his chutzpah. The famously independent-minded Paul Volcker, who was appointed to lead the Fed by Jimmy Carter and reappointed by Ronald Reagan, said: “The drug thing is a perfect example that he doesn’t adopt a conventional view. I think drug policy needs a new look and he’s been one of the people who say that.”

Soros’s money has been crucial in enabling him to voice maverick views: “That’s what led me to oppose Bush very publicly, because I was in a position that I could afford to do it,” he said. But he also believes his fortune and the automatic credibility it gives him in America has drawn the fire of conservative pundits such as Fox’s Bill O’Reilly and extremist pamphleteer Lyndon LaRouche. “Given the excessive esteem in which people who make money are held in America, I had to be demonised,” he said.

Their attacks worked. So much so that last year, as the Obama bandwagon gained speed and American financiers, along with much of the rest of the country, clamoured to jump on, his earliest heavyweight Wall Street backer kept a low profile. “Obama seeks to be a unifier,” Soros said. “And I have been a divisive figure because I’ve been demonised by the right. I thought my vocal support for him would not necessarily benefit him.”

. . .

At around 1.00am on November 5 2008, Soros sat on a peach-coloured sofa in his elegant Fifth Avenue apartment, with Queen Noor of Jordan to his left and Steve Clemons, of the New America think-tank, perched on the edge of a chair to his right. Around them milled a crowd of eclectic and jubilant guests, many still teary-eyed from Obama’s Grant Park victory speech, which had been broadcast on four flat-screen television sets in the apartment. Like most Soros soirées, the gathering included more artists and statesmen than Masters of the Universe: Michèle Pierre-Louis, the prime minister of Haiti and former head of her country’s Soros foundation; former World Bank chief James Wolfensohn; Volcker; and twentysomething Kwasi Asare, a hip-hop music promoter, were among the visitors.

Soros drank an espresso and, a few minutes later, a final champagne toast with the last of his guests. Alexander, his 23-year-old son, perched on the arm of his chair and ruffled his father’s hair in farewell. Everyone else took that as a signal to depart, too. Soros was in a mellow, triumphant mood that night – and with good reason. He had spotted Obama early on. His ubiquitous political consigliere, Michael Vachon, still has among his papers a rumpled itinerary from a trip he and Soros took to Chicago in February 2004. In the upper right-hand corner of the page, Vachon had scrawled, “Barack guy”. The Senate candidate had been keen to meet Soros and called the pair repeatedly during their visit. But it was a packed schedule and Soros could only offer a 7.30am breakfast slot at the Four Seasons.

Soros left that meal “very impressed”, a view that was confirmed when he read Obama’s autobiography and deemed him “a real person of substance”. A few months later, on June 7, Soros hosted a packed fundraiser for Obama’s Senate campaign at his upper east side home. Soros and his family contributed roughly $80,000, then the legal maximum.

Obama was impressing a lot of people at that time. But once it became clear that Hillary Clinton would be in the presidential race, nearly all of the established New York Democrats, particularly the older Wall Street crowd, lined up behind their local Senator and her machine, driven by a combination of loyalty and calculation. Dominique Strauss-Kahn, now the head of the IMF and then a possible French presidential candidate, said Soros told him in 2006 he was supporting “this young guy, Barack Obama. He was the first one to tell me this and he was right.” On January 16 2007, the day Obama formed a presidential exploratory committee, Soros contributed to his campaign and officially offered his backing. Before doing so, Soros called Hillary Clinton to let her know. “I look forward to your support in the general election,” she told him.

His decision to back Obama was consistent with his life-long affinity for moments of radical change. “I felt that America had gone so far off base that there was a need for discontinuity,” he said. As in the markets, Soros’s political bet on systemic transformation – his support for Obama, but also his early opposition to the war in Iraq and the “war on terror” – has come good.

For Soros, one happy consequence of now being in tune with the zeitgeist is that he is being taken seriously as a thinker on American public policy issues, particularly to do with the financial crisis. When he, along with the other four highest-earning hedge fund managers, testified before Congress in November, he was treated with respect and even deference – not the prevailing attitude towards billionaire financiers at the moment. Before Soros had even taken his coat off, he was greeted in the corridors by Democratic New York Congresswoman Carolyn Maloney. “Give him a nice office,” she told a staffer who was looking for a place where Soros could wait before his testimony. “He creates a lot of jobs in my district and supports a lot of good people.” After the hearing, a lawmaker and a staffer both approached Soros and asked him to autograph their copies of his book.

. . .

Being listened to on Capitol Hill, and by global policymakers more generally, is important to Soros. But what matters to him most of all – more than money, more than the political and social accomplishments of his foundation – is leaving an enduring intellectual legacy. He describes reflexivity as “my main interest”. Even as Soros met with increasing financial and public success through his fund and his foundation, he was deeply frustrated by his failure to be accepted as a serious thinker. He titled one chapter in his latest book “Autobiography of a Failed Philosopher”, and once delivered a lecture at the University of Vienna called “A Failed Philosopher Tries Again”. As a young man, he wanted to become an academic, but “my grades were not good enough”.

He writes that his first book, The Alchemy of Finance, was “dismissed by many critics as the self-indulgence of a successful speculator”. That reaction still prevails in some circles. Paul Krugman, the Nobel prize-winning economist, devotes half a chapter to Soros in his latest book, characterising him as “perhaps the most famous speculator of all times”. He also raises an eyebrow at Soros’s intellectual “ambitions”, tartly observing that he “would like the world to take his philosophical pronouncements as seriously as it takes his financial acumen”.

Another barrier to academic respectability is Soros’s self-confessed “phobia” of formal mathematics: “I understand mathematical concepts but I’m afraid of mathematical symbols, because you can easily get lost in them.” That fear proved no impediment to success in the quantitative world of finance, but it has hurt Soros’s street cred in economics departments. “Among academics, he suffers from the additional liability of not expressing it in the language of mathematics that has become fashionable,” Joe Stiglitz, another Nobel prize-winning economist, said. But Stiglitz believes his friend’s writing has become more current, partly thanks to the financial crisis: “By those economists interested in ideas, I think his work is taken seriously as an idea that informs their thinking.”

In the view of Larry Summers: “Reflexivity as an idea is right and important and closely related to various streams of existing thought in the social sciences. But no one has deployed a philosophical concept as effectively as George has, first to make money and then to change the world.”

Paul Volcker delivered a similar verdict: “I think he has a valid insight which is not always expressed as clearly by him as I might like.” Overall, he said, Soros is “an imaginative and provocative thinker … he’s got some brilliant ideas about how markets function or dysfunction.”

This is as close to mainstream intellectual acceptance as Soros has come in his two decades of writing and more than five decades since he gave up on academia. It feels like a breakthrough. When I asked him if he would still describe himself as a failed philosopher, he said no: “I think that I am actually succeeding as a philosopher.” For him, that is “obviously” the most important human accomplishment.

“I think it has to do with the human condition,” he said. “The fact that we are mortal and we would like to be immortal. The closest thing you can come to that is by creating something that lives beyond you. Wealth could be one of those things, but evidence shows that it doesn’t survive too many generations. However, if you can have an artistic or philosophical or scientific creation that withstands the test of time, then you have come as close to it as possible.”

Chrystia Freeland is the FT’s US managing editor

Click here to read an extract from George Soros’s e-book update to The New Paradigm for Financial Markets – The credit crisis of 2008 and what it means