What to do if you don't start working right away maybe 3 years later after graduating from nursing school.Is it difficult to be hired?

I would advise you to take a refresher course and then apply for new grad programs at hospitals.

Yes it will be much harder to get a nursing job. New grads have a high learning curve and after 3 years of no experience, it will be even higher.