Of course the short answer to the question is: English conversations are so difficult because they are fast.
In a real conversation, you have no time.
You have no time to think about grammar.
You have no time to think about pronunciation.
You have no time to think about vocabulary.
You must understand everything instantly, and you must respond immediately and correctly. All of this must happen automatically.
Textbooks and schools simply don’t teach you those skills.
Take a look to this video and find more.