Quick answer
ir = to go, movement away from where the speaker is.
venir = to come, movement towards where the speaker is right now.
Rule of thumb: venir only points at the speaker. If you are moving to where your listener is, Spanish still uses ir.
Side by side
Memory trick
Before you pick the verb, ask: does the movement end where I am standing as I speak? Only then is it venir. Everything else is ir.
Typical mistakes
Vengo a tu casa esta noche.
Voy a tu casa esta noche.
You are not at your friend's house yet, so the movement goes away from you. Spanish needs ir even though English says "I am coming over".
Ya vengo, espérame en la puerta.
Ya voy, espérame en la puerta.
Someone calls you and you set off towards them. From your own position that is movement away, so ya voy. Ya vengo exists but means "I'll be right back".
Check yourself
Your friend calls from the restaurant. You answer: "Ya ___, cinco minutos."