In Horde, starting a new wave clears all the weapons dropped by enemies so if you want to carry over heavy weapons such as mulchers boomshields or mortars, pick them up and carry them until the new wave of enemies appear. This way, you can still get a new copy of the weapon if it is one of the default weapons of the map or have the advantage of having extra firepower at the start of the wave.
