Numbers
Not only this game has a special language, but it also has an interesting number system. Thank the Zu! The number system is also based on a cube.
Source: Jules Cárdenas on SketchFab
The number system starts with an empty square to show zero, and a single line at a position similar to 12 o'clock to show the number one. Rotating it clockwise would show subsequently 2, 3 and 4.
After that, some numbers can be written in more than one way, based on how it is summed: 5 can be written summing 2 and 3, or 1 and 4. 6 can be either 2 plus 4, or 1 + 2 + 3. To be fair, 3 can be written summing 1 and 2 as well, but look at the graphics, they are easier to understand now that you know the logic.
The counting artifact is a perfect cube showing the 6 types of symbols, and they can be rotated to make the different numbers. The numbers 1, 2, 3 and 4 can be written using the same face, numbers 4 and 6 too. Numbers 3, 5 and 7 also share a face, and numbers 6, 7, 8 and 9 share a single face as well.
This image shows the net of the counting cube, before we actually find it in the interiors of a certain building; and the image below, in the same original room, it explains visually how the sum works.
This info will be especially useful at the bell tower, cemetery and other puzzles that require you to use a certain order for whatever inputs they are asking.
There is also another clue in the the aforementioned room, showing some stacking squares, dots and lines:
If you remember the powers of two, 20 equals 1 (a dot or point), 21 is 2 (represented by a line), 22 is 4 (a square!) and 23 is 8 (a cube!). The image shows to a society that is not quite used to cubes that yes, the dimensions can go up. There is more than 2D, and they represented it using two planes.
Reality is perception. Perception is subjective.
There is logic by using power of two. Most computers operate with bits and bytes, the information comes in binary - either a 0 or a 1. This requires power of two, and matches the themes on FEZ. That's why so many things in this game will operate in numbers that are powers of two, especially things related with number 8. You need 8 cube bits to form a golden cube, for example, and as you acquire a cube bit, a small tune will play - each in a slighly higher note than the other, forming an octave (C - D - E - F - G - A - B - C). The doors you unlock with cubes are also powers of two, and the amount of cubes to either endings (32 and 64) are then 25 and 26, respectively.
Some fans comment that the numeral system found in the FEZ game is very similar to the one in Myst, called the D'ni, and possibly inspired in that system. However, the system created by the Zu is a bit more straightforward and compatible with pixel art.