Grayling tells Work providers "that's tough"