For a continuous curve of families of Dirac type operators we define a higher spectral flow as a K-group element. We show that this higher spectral flow can be computed analytically by n-forms and is related to the family index in the same way as the spectral flow is related to the index. We introduce a notion of Toeplitz family and relate its index to the higher spectral flow. Applications to family indices for manifolds with boundary are also given.