Answered by Sayyidi Habib Umar bin Hafiz (may Allah protect him and benefit us by him)
How do we understand the statement of Shaykh Abu Madyan that someone who is not on the same path as you is like a child even if he is seventy years old?
This refers to someone who is not trying to travel to Allah by acting upon the Sacred Law, someone who does not help you to obey Allah and avoid disobeying Him. You should avoid keeping the company of such a person, just as you would avoid keeping the company of a small child.
It does not mean that you should avoid keeping the company of someone who does not belong to the same spiritual path or order (tariqah) as you. Shaykh Abu Madyan himself kept the company of people from different spiritual paths and many people from different spiritual paths kept his company.