Tomba must be in the Minitta state, either by using the Minitta Mushroom or Minitta Machine, in order to enter the Mouse Village via the door in Town of the Fishermen. An Adventurer's Chest can be found here and two million Adventure Points is needed to open it.