We present a systematic algorithm which tests for the existence of collective self-organization in the behavior of agents in social systems, with a concrete empirical implementation on the Dow Jones Industrial Average index (DJIA) over the 20th century and on the Hong Kong Hang Seng composite index (HSI) since 1969. The algorithm combines ideas from critical phenomena, the impact of agents’ expectations, multiscale analysis, and the mathematical method with pattern recognition of sparse data. Trained on the three major crashes in DJIA of the century, our algorithm exhibits a remarkable ability for generalization and detects in advance 8 other significant drops or changes of regimes. An application to HSI gives promising results as well. The results are robust with respect to the variations of the recognition algorithm. We quantify the prediction procedure with error diagrams.