We present a systematic algorithm which tests for the existence of collective self-organization in the behavior of agents in social systems, with a concrete empirical implementation on the Dow Jones Industrial Average index (DJIA) over the 20th century and on the Hong Kong Hang Seng composite index (HSI) since 1969. The algorithm combines ideas from critical phenomena, the impact of agents’ expectations, multiscale analysis, and the mathematical method with pattern recognition of sparse data. Trained on the three major crashes in DJIA of the century, our algorithm exhibits a remarkable ability for generalization and detects in advance 8 other significant drops or changes of regimes. An application to HSI gives promising results as well. The results are robust with respect to the variations of the recognition algorithm. We quantify the prediction procedure with error diagrams.

We introduce a novel non-parametric methodology to test for the dynamical time evolution of the lag–lead structure between two arbitrary time series. The method consists in constructing a distance matrix based on the matching of all sample data pairs between the two time series. Then, the lag–lead structure is searched as the optimal path in the distance matrix landscape that minimizes the total mismatch between the two time series, and that obeys a one-to-one causal matching condition. We apply our method to the question of the causality between the US stock market and the treasury bond yields and confirm earlier results on a causal arrow of the stock markets preceding the Federal Reserve Funds adjustments as well as the yield rates at short maturities in the period 2000–2003. The application to inflation, inflation change, GDP growth rate and unemployment rate unearths non-trivial causal relationships: the GDP changes lead inflation especially since the 1980s, inflation changes lead GDP only in the 1980 decade, and inflation leads unemployment rates since the 1970s. In addition, we detect multiple competing causality paths in which one can have inflation leading GDP with a certain lag time and GDP feeding back/leading inflation with another lag time.

We present a general methodology to incorporate fundamental economic factors to the theory of herding developed in our group to describe bubbles and antibubbles. We start from the strong form of rational expectation and derive the general method to incorporate factors in addition to the log-periodic power law (LPPL) signature of herding developed in ours and others’ works. These factors include interest rate, interest spread, historical volatility, implied volatility and exchange rates. Standard statistical AIC and Wilks tests allow us to compare the explanatory power of the different proposed factor models. We find that the historical volatility played the key role before August of 2002. Around October 2002, the interest rate dominated. In the first six months of 2003, the foreign exchange rate became the key factor. Since the end of 2003, all factors have played an increasingly large role. However, the most surprising result is that the best model is the second-order LPPL without any factor. We thus present a scenario for the future evolution of the US stock market based on the extrapolation of the fit of the second-order LPPL formula, which suggests that herding is still the dominating force and that the unraveling of the US stock market antibubble since 2000 is still qualitatively similar to (but quantitatively different from) the Japanese Nikkei case after 1990.

Following a long tradition of physicists who have noticed that the Ising model provides a general background to build realistic models of social interactions, we study a model of financial price dynamics resulting from the collective aggregate decisions of agents. This model incorporates imitation, the impact of external news and private information. It has the structure of a dynamical Ising model in which agents have two opinions (buy or sell) with coupling coefficients, which evolve in time with a memory of how past news have explained realized market returns. We study two versions of the model, which differ on how the agents interpret the predictive power of news. We show that the stylized facts of financial markets are reproduced only when agents are overconfident and mis-attribute the success of news to predict return to herding effects, thereby providing positive feedbacks leading to the model functioning close to the critical point. Our model exhibits a rich multifractal structure characterized by a continuous spectrum of exponents of the power law relaxation of endogenous bursts of volatility, in good agreement with previous analytical predictions obtained with the multifractal random walk model and with empirical facts.

Using a methodology developed in previous papers, we analyze the quarterly average sale prices of new houses sold in the USA as a whole, in the Northeast, Midwest, South, and West of the USA, in each of the 50 states and the District of Columbia of the USA, to determine whether they have grown at a faster-than-exponential rate which we take as the diagnostic of a bubble. We find that 22 states (mostly Northeast and West) exhibit clear-cut signatures of a fast-growing bubble. From the analysis of the S&P 500 Home Index, we conclude that the turning point of the bubble will probably occur around mid-2006.