I just preload EVERYTHING (just a scene with everything in it that loads before the main scene) haha. Particle emitters I found were the real culprit, the first time a particle is emitted it causes like a 2 second freeze... That's for EVERY particle system. I put them in my preload scene with autofire enabled and that did the trick.