Root Mean Square Error (RMSE) is the standard deviation of the residuals (prediction errors). Residuals are a measure of how far from the regression line data points are; RMSE is a measure of how spread out these residuals are. In other words, it tells you how concentrated the data is around the line of best fit.
A loss function is a measure of how good a prediction model does in terms of being able to predict the expected outcome.
To know more about RMSE and Loss refer to following links: