"Turtles all the way down" is an expression of the problem of infinite regress. The saying alludes to the mythological idea of a World Turtle that supports a flat Earth on its back. It suggests that this turtle rests on the back of an even larger turtle, which itself is part of a column of increasingly larger turtles that continues indefinitely.
The saying holds that the world is supported by an infinite stack of increasingly larger turtles.
Four World Elephants resting on a World Turtle
William James
The World Turtle, also called the Cosmic Turtle or the World-bearing Turtle, is a mytheme of a giant turtle supporting or containing the world. It occurs in Hindu mythology, Chinese mythology, and the mythologies of some of the indigenous peoples of the Americas. The comparative mythology of the World-Tortoise discussed by Edward Burnett Tylor (1878:341) includes the counterpart World Elephant.
An 1877 drawing of the world supported on the backs of four elephants, themselves resting on the back of a turtle.