Latex foam has a higher elasticity or resiliency than memory foam. The result is that latex will feel more springy and memory foam will feel more solid. Many companies are blending both of these materials for the perfect mattress combination. Therefore, memory foam will not push back up against your body when pressure is applied to the pillow.
Rare allergies to latex can be triggered, but usually upon skin contact. Because there are no chemicals used in latex composition, it is the ILD only that determines the feel of this product.