Three types of rock specimens, three point bending specimen, anti symmetric four-point bending specimen and direct shearing specimen, were used to achieve Mode Ⅰ, Mode Ⅱand mixed mode Ⅰ2 Ⅱfracture, respectively. Microscopic characteristics of the three fracture modes of brittle rock were studied by SEM technique in order to analyze fracture behaviors and better understand fracture mechanisms of different fracture modes of brittle rock. Test results show that the microscopic characteristics of different fracture modes correspond to different fracture mechanisms. The surface of Mode Ⅰfracture has a great number of sparse and steep slip steps with few tearing ridges and shows strong brittleness. In the surface of Mode Ⅱfracture there exist many tearing ridges and densely distributed parallel slip steps and it is attributed to the action of shear stress. The co action of tensile and shear stresses results in brittle cleavage planes mixed with streamline patterns and tearing ridges in the surface of mixed mode Ⅰ Ⅱfracture. The measured Mode Ⅱfracture toughness KⅡC and mixed mode Ⅰ Ⅱfracture toughness KmC are larger than Mode Ⅰfracture toughness KⅠC. KⅡC is about 3.5 times KⅠC, and KmC is about 1.2 times KⅠC.

The study of stress state under shear load indicated that shear load applied to crack induces tensile fracture instead of shear fracture. It leads to consider that loading mode does not always correspond to fracture mode. Under different mixed loading, fracture belongs to single tensile fracture in most cases, only in certain special case shear fracture can occurs. Thus fracture mode should be distinguished from loading mode. To judge what fracture mode will occur under arbitrary loading prerequisites for mode I and mode II fracture become vital important. The criterion to define mode I or mode II fracture under any plane loading can be summarized as: prerequisites for mode I fracture are: 1), fθmax>frθmax or frθmax/fθmax<1, or 2), frθmax/fθmax>1, but frθmax/fθmax<KIIC/KIC; prerequisite for mode II fracture is frθmax/fθmax>1 and frθmax/fθmax>KIIC/KIC, where fθmax is the dimensionless maximum circumferential stress intensity factor and frθmax is the dimensionless maximum shear stress intensity factor.

Mode Ⅱ fracture initiation and propagation plays an important role under certain loading conditions in rock fracture mechanics. Under pure tensile, pure shear, tension-and compression-shear loading, the maximum Mode Ⅰ stress intensity factor, KI max; is always larger than the maximum Mode Ⅱ stress intensity factor, K Ⅱ max: For brittle materials, Mode Ⅰ fracture toughness, KIC; is usually smaller than Mode Ⅱ fracture toughness, KIIC: Therefore, KI max reaches KIC before K Ⅱ max reaches KIIC; which inevitably leads to Mode I fracture. Due to inexistence of Mode Ⅱ fracture under pure shear, tension-and compression-shear loading, classical mixed mode fracture criteria can only predict Mode Ⅰ fracture but not Mode II fracture. A new mixed mode fracture criterion has been established for predicting Mode Ⅰ or Mode Ⅱ fracture of brittle materials. It is based on the examination of Mode Ⅰ and Mode Ⅱ stress intensity factors on the arbitrary planey; KIðyÞ and KIIðyÞ; varying with yð 180 pypþ 180 Þ; no matter what kind of loading condition is applied. Mode Ⅰ fracture occurs when ðKII max=KⅠ maxÞo1 or 1oð K Ⅱ max=KI maxÞoðKⅡ C=KICÞ and KI max ¼ KIC at yIC: Mode Ⅱ fracture occurs when ðKⅡ max=KI maxÞ>ðKIIC=KICÞ and KⅡ max ¼ KIIC at yIIC: The validity of the new criterion is demonstrated by experimental results of shear-box testing. Shear-box test of cubic specimen is a potential method for determining Mode Ⅱ fracture toughness KIIC of rock since it can create a favorable condition for Mode Ⅱ fracture, i.e. KⅡ max is always 2-3 times larger than KⅠ max and reaches KⅡC before KⅠ max reaches KⅠC: The size effect on KⅡC for single-and double-notched specimens has been studied for different specimen thickness B; dimensionless notch length a=W (or 2a=W) and notch inclination angle a: The test results show that KⅡC decreases as B increases and becomes a constant when B is equal to or larger than W for both the single-and double-notched specimens. When a=W (or 2a=W) increases, KⅡC decreases and approaches a limit. The a has a minor effect on KIIC when a is within 65-75°: Specimen dimensions for obtaining a reliable and reproducible value of KIIC under shear-box testing are presented. Numerical results demonstrate that under the shear-box loading condition, tensile stress around the notch tip can be effectively restrained by the compressive loading. At peak load, the maximum normal stress is smaller than the tensile strength of rock, while the maximum shear stress is larger than the shear strength in the presence of compressive stress, which results in shear failure.