They/we simply don't know how and don't have the muscles for it either. This is especially true of stoop labor, the backbreaking jobs of growing and harvesting our fruits and vegetables.
Sure, a lot of backbreaking manual labor went in to running an American family farm in Ye Old Days, and it was taken for granted. But why do you think people left those farms by the millions when they had other opportunities? That exodus started after WWI, just about a century ago.
Since then, farms have been increasingly consolidated by agribusiness, and when they hire people they have to look for those who come from rural areas themselves and have the knowhow. Those people have to be willing, and there are plenty of places south of the border where life is so hard that even American farm labor (or chicken-packing) looks better than what they have. Sorry, it was ever thus.
I don't think the American worker is "too lazy" to do farm work. I think our workers will work hard at what they know. I don't think they are much inclined to travel far away from what they are familiar with in cities, all for the privilege of doing backbreaking work 6 days a week, and sleeping in their cars or sheds or thoroughly substandard housing, and gettng a thoroughly substandard wage.
PS: In case it's not clear, I believe immigrant farm workers are badly treated and deserve better. But refusing them entry is not the answer.