In the present paper, the surface and size effects on the alloying ability and phase stability of immiscible alloy nanoparticles have been studied with calculating the heats of formation of Au-Pt alloy nanoparticles from the single element nanoparticles of their constituents (Au and Pt) with a simple thermodynamic model and an analytic embedded atom method. The results indicated that, besides the similar compositional dependence of heat of formation as in bulk alloys, the heat of formation of alloy nanoparticles exhibits notable size-dependence, and there exists a competition between size effect and compositional effect on the heat of formation of immiscible system. Contrary to the positive heat of formation for bulkimmiscible alloys, a negative heat of formation may be obtained for the alloy nanoparticles with a small size or dilute solute component, which implies a promotion of the alloying ability and phase stability of immiscible system on a nanoscale. The surface segregation results in an extension of the size range of particles with a negative heat of formation. The molecular dynamics simulations have indicated that the structurally and compositionally homogeneous Au Pt nanoparticles tend to form a core-shell structure with temperature increasing.