To understand the answer to this question, it's important to understand what filters actually do.
A common misconception about filters is that they are meant to keep the tank clean. This is not entirely true. While mechanical filters can suction up floating debris, most debris falls into the substrate and decays. The filter is meant to siphon up water and run it through media with beneficial bacteria, returning cleaner water to the tank. This is called biological filtration and is part of the nitrogen cycle.

What is the nitrogen cycle?

• The nitrogen cycle starts with ammonia (hydrogen nitride) which comes from fish waste, decaying food, and decaying plant matter. Ammonia in large amounts is toxic.
  
• Beneficial bacteria converts ammonia to nitrites. Nitrite is toxic.
• Beneficial bacteria converts the nitrites into nitrates. Nitrates are less toxic to fish, and plants even use them for food.
• You must remove excess nitrates from the tank by doing regular water changes.

If you don't have a filter, the beneficial bacteria can exist in the substrate, but there would be no water flow to carry the toxins to them.
In this case, you'd need to do large water changes multiple times a week to manually remove the toxins.
This is why filters are recommended for all tanks.