Fibrillar silicate (FS)/rubber nanocomposites were successfully prepared by directly mixing modified FS with rubber matrix. It is found that FS could be separated into nano-fibrils with diameters less than 100 nm by the shear forces during mixing. The stress-strain characteristics of these composites are similar to those for short micro-fiber/rubber composites (SFRC). Nevertheless, these FS/rubber composites have some outstanding advantages over the conventional SFRC, even though the reinforcing effect of FS is restricted due to its small shape aspect ratio. More importantly, the differences in mechanical properties of the composites in the two different directions show that SBR/FS and NBR/FS composites both exhibit obvious anisotropy, which strongly depends on the preparation process, FS concentration, and rubber matrix. These factors were thoroughly investigated in this paper, and it can be concluded that the anisotropy of the composites was due to the orientation of nano-fibrils.